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Introduction 



One of the most difficult challenges to effective substance abuse prevention is finding the right balance 
between maintaining the fidelity of a science-based model prevention program and promoting adapta- 
tion of that program to reflect the circumstances of the community where it is being implemented. 

The Center for Substance Abuse Prevention, through its National Center for the Advancement of 
Prevention, is taking a leadership role in (1) synthesizing the knowledge base on this important sub- 
ject — through this literature review and related products targeted to specific audiences; and (2) mobi- 
lizing those audiences — program developers, the prevention practitioners and community leaders who 
implement the programs, researchers, funders, and public policymakers — to address the need for bal- 
ance in more rigorous and practical ways. The better we understand the complexities of fidelity and 
adaptation balance, the more strategically we can approach these issues in our work — and the better the 
prevention field as a whole can deal with the challenges of balancing program fidelity and adaptation. 

The report that follows is an important part of that process. This “ state- of-the-arf’ review, prepared for 
CS AP, surveys 125 published and unpublished studies related to fidelity and adaptation balance, span- 
ning more than 25 years. An extensive list of references guides researchers to the full body of literature 
surveyed for this review. 

The report first defines several key terms and then reviews the relevant research in some detail. It then 
presents several conclusions drawn from the literature review. The fundamental conclusion is that 

attention to BOTH program fidelity and adaptation during the complex process of program 
implementation is critical to successful, sustained implementation of science-based substance 
abuse prevention programs. 

In addition, this paper proposes an initial set of guidelines for program implementers and also several 
unresolved issues that require attention from each of the primary ‘audiences for this work. The paper 
concludes with a consideration of next steps for CS AP and others in order to advance the understand- 
ing of program fidelity and adaptation balance in substance abuse prevention. 



I 



2002 Conference Edition 



DHHS/SAMHSA/CSAP 



Program Fidelity and Adaptation in Substance Abuse Prevention 



N 

Why Program Fidelity and 
Adaptation Are Important 

Program developers and prevention researchers are legitimately concerned that changes in a science- 
based program will dilute or even dissipate its effectiveness. Community leaders and prevention prac- 
titioners are equally concerned that “not one size fits all.” The inability to modify programs may pro- 
duce local resistance; or worse, rigid fidelity may lead to programs that are irrelevant or even inappro- 
priate for meeting community needs. Policymakers and funders struggle with defining what require- 
ments for fidelity or permissions for adaptation are appropriate in guidance related to funding or pub- 
lic policy. 

Each of these groups can contribute to the improved ability of the prevention field as a whole to deal 
with the challenges of balancing program fidelity and adaptation. For instance, program developers and 
prevention researchers can use scientific analysis to identify “core components” of effective and model 
prevention programs — those elements that must be maintained rigorously in order for the program to 
work. Community leaders and prevention practitioners can develop program implementation approach- 
es that address fidelity/adaptation balance strategically. 

Policymakers and funders can build improved standards for fidelity/adaptation balance into grant mak- 
ing and public policy about prevention programming. And all these groups can work together in creat- 
ing partnerships to change the culture of prevention, so that fidelity/adaptation balance issues can be 
addressed more effectively when implementing science-based programs in the field. 

In a sense, all these groups want to know “what the boundaries are” (Emshoff et al., under review). 
There is considerable scientific evidence that many science-based prevention programs still produce 
positive results despite significant adaptation. Some adaptations are, in fact, necessary for program suc- 
cess, given widely varying circumstances in different organizations or communities. 

However, there is also significant evidence that the greatest impact from these programs results when 
there is program fidelity with respect to certain key elements. And some adaptations are undesirable, 
whether deliberate or accidental. 

Efforts both to promote fidelity and to engage in adaptation already happen all the time, of course, but often 
not as strategically or effectively as would be possible with better guidance. The conclusions and guidelines 
that emerge from the literature review can help provide a better balance between these concerns. 
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Examples of Fidelity/ Adaptation 
Balance 

from the “Real World” 
of Program Implementation 

In Life Skills Training (LST), a well-validated program developed by Gilbert Botvin at Cornell 
University (Botvin et al, 1995, 1989), one component involves middle school students talking in their LST 
sessions about tobacco advertisements. LST’s curriculum suggests that students cut out tobacco ads at home 
and bring them to class for discussion. Sometimes teachers will also ask students to draw their own humor- 
ous ads and share them during the LST sessions. This is an example of program adaptation (specifically, an 
enhancement, since it adds something to LST that does not substitute for any existing component). 

Some teachers drop the LST curriculum session on relaxation, because they fear losing control of their 
students during this enforced “quiet time.” Others drop sessions because of time constraints. LST is an 
eight-session program; but in some schools, only six sessions are available for completing the program. 

These adaptations can have a significant bearing on whether the LST experience of some students is really 
the same as that of others. For LST, as for all science-based substance abuse prevention programs used 
throughout the United States, having a more unified set of guidance for fidelity/adaptation balance (and 
indeed, for the entire process of program implementation) and verifying that the implementation was effec- 
tive would likely improve overall program impact (G. J. Botvin, personal communication, June 2000). 

As with other substance abuse prevention programs, many LST implementers are analyzing the program 
to determine what they believe are its core components, selecting ones that work for them, deleting those 
that do not, and adding elements they think will improve the program. This program adaptation will go 
on, says the weight of evidence, regardless of what program developers or researchers think about it. 

Likewise, program implementers as well as developers have genuine interest in fidelity. For instance, 
in CSAP’s Model Program Replication grant initiative, grant project directors voiced concern about 
keeping faithful to the model they were implementing — an interest also expressed by the funder — and 
they had help from program developers in doing so (Emshoff et al., under review). 

The Emshoff study also reported an especially relevant project experience. The program developer 
pushed for fidelity to a particular aspect of the program that was initially resisted by the implementer 
grantee. Later, however, the grantee acknowledged that this very component was a significant part of 
the program’s success in its replication setting. 
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Overview of This Paper 

The following literature review examines the impact of program fidelity and adaptation activities dur- 
ing the implementation of substance abuse prevention programs, such as the examples above. The 
review includes research on both individual programs and meta-analyses of many such programs. 

The research confirms that there are real impacts from the implementation activities explored here. For 
instance, Tobler and Stratton (1997), in one of the largest meta-analyses of school-based substance abuse 
prevention programs, conclude that “the large decreases in effectiveness experienced when delivered on 
a large scale suggests factors other than statistical leveling of effect sizes. ...Implementation factors pro- 
vide a more probable explanation and can be a crucial mediating factor in determining success” (p. 114). 

The resulting challenges for effective program implementation are not confined to substance abuse pre- 
vention programs. Domitrovich and Greenberg (2000) review the body of work on disseminating and 
implementing effective preventive interventions in children’s mental health, violence prevention and 
school safety, social-emotional learning, and positive youth development — as well as substance abuse 
prevention. They conclude: 

Surprisingly, many of the highest quality programs fail to take adequate steps to mon- 
itor and verify program integrity. This weakens the conclusions that can be drawn 
regarding the program outcomes and reduces the likelihood that replications will 
resemble the original program, (p.2) 

They also comment about adaptation issues, concluding that in most implementation circumstances, “it 
will be necessary from the outset to make location adaptations. Further, over time it is likely that addi- 
tional changes will be made by local program deliverers” (p. 23). 

This conclusion emphasizes the importance of not viewing issues of program fidelity and adaptation in 
an “us-them” conflict between developers and implementers. In fact, program developers are as likely 
to see the value of carefully executed adaptation as implementers are to see the importance of fidelity. 
This suggests, importantly, that implementers (prevention agencies and communities) need to be 
involved at all stages of program development, offering input from their real-world perspective; and 
developers need to be involved in all aspects of implementation: providing technical assistance and 
learning from this involvement about improvements that can be made in their programs. 

Exhibit 1 shows a conceptual model for stages of program implementation. In this model, implementation is 
preceded by activities of program development, validation, and dissemination. Upon completion of imple- 
mentation, often there is feedback to program developers, which may result in modification of the original 
program (not just of one implementation of it). Between these pre- and post- activities, there are seven stages 
of program implementation. This model serves as a framework for organizing this literature review. This 
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State-of-the-art review provides a context for appraising the current status of the field. It includes: (a) 
key theoretical and empirical studies that have contributed to the present state of knowledge, (b) review 
papers that provide access to the larger literature, and (c) work specific to substance abuse prevention, 
with some attention to work from health and education. 



EXHIBIT I: 

STAGES IN SUBSTANCE ABUSE PREVENTION PROGRAM IMPLEMENTATION 

(Program DevelopmentA^alidation/Dissemination) 

Program Adoption 

Needs and Assets Assessment 

Readiness Assessment 

Program Assessment 
Theory and/or Logic Model 
Core Components Analysis 
Developer Consultation 

Program Implementation 
Fidelity/ Adaptation Balance 
Implementation Process 
Implementation Setting 
Client Characteristics 
Larger Context 

Evaluation 
Fidelity Instrument 
Process Evaluation 
Outcome Evaluation 

Sustainability 

Revisiting Fidelity/Adaptation 
Routinization 

(Feedback to Developer/Program Modification) 
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Definitions 



Terminology varies within the body of research on fidelity and adaptation and related issues. Here are 
definitions for key terms used in the CSAP studies and in the balance of this paper: 

Program: an intervention for preventing substance abuse, usually defined in a manual or curriculum. 
The emphasis in this paper is on “science-based programs,” which are defined below. Programs are a 
sub-set of the more general term “innovation,” which includes many other kinds of ideas or technolo- 
gies that get implemented by people, organizations, or communities. The latter term is used in much of 
the wide-ranging literature discussed here. 

Program Fidelity: the degree of fit between the developer-defined components of a substance abuse 
prevention program and its actual implementation in a given organizational or community setting. 

In other words, how well does a promising, effective, or model program match the specifications of the 
original? The program’s elements are specified in a program manual, curriculum, or core components 
analysis (defined below). In essence, the developer provides a “recipe” for replicating the program, and 
also describes the fidelity instrument(s) (discussed below) for measuring the “fit,” or precision, of its 
implementation. 

What we call “fidelity” here also has been called program “adherence” or “integrity” in some of the lit- 
erature on this subject. Terms from medicine also have been used to discuss the overall degree of fideli- 
ty (without necessary reference to program components) — “dosage,” “strength of treatment” or “inten- 
sity” (Boruch & Gomez, 1977). The latter terms all refer to the amount of a program delivered — what 
M. A. Pentz (personal communication, April 2001) calls “exposure.” In this paper, exposure will be 
considered part of the larger concept of program fidelity. 

The medical term “compliance” means the extent to which a particular protocol or regimen is followed. 
Compliance as a metaphor for fidelity/adaptation balance will be considered later in this paper. 

Program Adaptation: deliberate or accidental modification of the program, including the following: 

a. deletions or additions (enhancements) of program components; 

b. modifications in the nature of the components that are included; 

c. changes in the manner or intensity of administration of program components called 
for in the program manual, curriculum, or core components analysis; or 

d. cultural and other modifications required by local circumstances. 
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Sometimes there are changes in the target populations for which the program was intended, or changes 
in the training (Palumbo, Maynard-Moody, & Wright, 1984; Mayer, Blakely, & Davidson, 1986). 
“Mutual adaptation,” a term introduced into the literature by Berman and McLaughlin (1978) in a major 
study of educational innovations (described further below), involves adaptation both of the innovation 
and of the organization or community in which it is implemented. 

Adaptation also is sometimes referred to as “reinvention” (Rogers, 1995a), and more will be said about 
this term later. Some writers have referred to “cosmetic” adaptation, as when only the name or some 
superficial element of a program is changed in order to promote local ownership. And “accidental” 
adaptation can be superficial or significant, involving changes that implementers do not realize they 
have made (often because of insufficient technical assistance on implementation). 

Fidelity/Adaptation Balance: a dynamic process, often evolving over time, by which those involved 
with implementing a science-based substance abuse prevention program address both the need for 
fidelity to the original program and the need for local adaptation. 

This balance is achieved at the point of full program implementation, but is not necessarily stable over 
time. Strategically revisiting fidelity/adaptation balance, in fact, may be an important element in pro- 
gram sustainability, as discussed later. 

Core Components: those elements of a program that fundamentally define its nature, and that analy- 
sis — from theory, from a logic model, and/or from empirical evidence — shows are most likely to 
account for its main effects. 

Some programs have essentially only their core components. Others have discretionary or optional 
components, which can be deleted without major impact on the program’s effectiveness, or which are 
not essential for the program’s main target audience. 

Core components are the essential, or main, ingredients in the recipe. CSAP is currently conducting a 
core components analysis of effective and model programs, which is expected to be completed in early 
2002, as part of developing and maintaining the National Registry of Effective Prevention Programs 
(NREPP). 

Implementation: the complex process by which a substance abuse prevention program is put into place 
in a community or organization, for use with a particular target audience. 

As indicated in the Program Implementation Stages (Exhibit 1), this is a multi-stage process. The first 
step is program adoption — the initial decision to implement made by the implementing organization or 
community. The related sequence of events ideally (though not always) includes some sort of initial 
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assessment of needs and assets in the local community, and an assessment of the readiness of the organ- 
ization or community for implementation to happen. (Readiness can have a particularly important 
impact on the ultimate success of the implementation effort.) 

An assessment of the program itself also can contribute to effective implementation — by helping imple- 
menters understand its theory base, by adopting (or constructing) a logic model that presents graphi- 
cally how the program operates, and/or by interpreting (or in some cases conducting) a core compo- 
nents analysis to determine the fundamental aspects of the program. 

The actual program implementation includes several sub- stages: 

• striking the balance between program fidelity and adaptation 

• the sequence of steps needed to install the program fully 

• fitting the implementation to the organizational or community setting 

• fitting the implementation to the characteristics of the population to be addressed 

• fitting the implementation to the larger context (e.g., other important aspects of the 

community, the current local or national climate for substance abuse prevention, etc.) 

Ideally, the programs that continue to operate are those that, as shown through evaluation, continue to 
meet identified needs, are cost-effective, and lack significant side effects. If the program is successful 
and continues to operate, implementation is followed by some sort of movement to long-term sustain- 
ability, perhaps including routinization as a part of the host organization or community, (e.g., a line item 
in the budget for operating the program, etc.). 

This model for program implementation stages is an ideal. However, not all prevention programs are 
implemented following a systematic assessment; not all are evaluated. And though the list of stages sug- 
gests a linear order, the occurrence of activities may vary from one implementation to another. 

Furthermore, definitions for what implementation includes vary within the literature. Rohrbach, 
Graham, and Hansen (1993), for example, define implementation as including both quantity and qual- 
ity elements. They interpret these as program exposure (the number of program components presented 
and the amount of time spent on each), and program fidelity (measures of the extent to which a pro- 
gram was implemented as designed). M. A. Pentz (personal communication, April 2001) also breaks out 
exposure and fidelity (adherence to program content, methods, procedures, etc.) and also includes 
strategic reinvention of the program to increase its adherence in a particular environment. 
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Here, the definition used separates program fidelity and adaptation conceptually. It also blends expo- 
sure and fidelity concepts together in a larger frame. Finally, by this definition, program fidelity/adap- 
tation is considered just one element of a larger sequence of implementation stages. 

Sdence>Based Programs: CSAP defines science-based substance abuse prevention programs as those 
programs that are theory-driven, have program activities that are related to the underlying theory, and 
have been reasonably well implemented and evaluated in field settings. As indicated in Exhibit 2, CSAP 
divides science-based programs into three categories: 

• effective programs — programs that have scientific evidence they consistently 
achieve positive outcomes 

• model programs — effective programs that CSAP has made available for 
systematic dissemination, and for which technical assistance to field 
implementers can be obtained from the program’s developers 

• promising programs— progmms that show some evidence of positive outcomes, 
but are not yet ready to be classified as effective programs. 
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Exhibit 2 



Science-Based Programs 

Theory-Driven 

Program Activities Related to Theory 
Reasonably Well Implemented & Evaluated 




Promising 

Some Positive 
Outcomes 




Effective 



Consistently Positive Outcomes 
Strongly Implemented & Evaluated 




— V 

Model 

• Availability for Dissemination 

• Technical Assistance Available 
from Program Developers 
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A Resurgence in Interest 



Issues of fidelity/adaptation balance related to program implementation have been studied and dis- 
cussed since at least the 1960s, when applied research in the education and human services fields 
became more widespread, and the field of program evaluation was bom. The impetus came from efforts 
to explain why so many evaluation results with previously validated programs were null or inconsis- 
tent — raising the possibility of an effect from the implementation of the program, rather than the pro- 
gram itself. 

In recent years, there has been renewed interest in this subject. For one reason: there simply are more 
effective substance abuse prevention programs available, resulting in quantitatively greater need to look 
at fidelity /adaptation concerns. CSAP is generating one central collection of these programs in the 
National Registry of Effective Prevention Programs (NREPP). Dusenbury and Falco (1995) and Elliott 
(1998) are examples of other collections. 

Research over the last 40 years also has helped us to understand better the complexities of implemen- 
tation for educational and substance abuse programs. This leads to greater chances for effective inter- 
vention. Technologies for managing the implementation process are now available,' some of them guid- 
ed by sophisticated behavioral and management science concepts. 

Furthermore, substance abuse prevention programs are subjected to more change, more complexity, and 
more demands for producing concrete results, while at the same time, substance use persists for certain 
populations (Backer, 2000). This leads to greater urgency for communities to learn from studies in this field. 

Funders are demanding more program accountability (which is easier to do with science-based pro- 
grams), such as with the nearly 100 grant projects funded under CSAP’s Family Strengthening initia- 
tive, and the call for science-based programs used as part of CS AP’s State Incentive Grant funding. The 
Department of Education’s “Principles of Prevention” mandates similar attention to science-based pro- 
grams for its grant competitions. And the U.S. Congress now advocates that Drug Free Schools and 
Communities funds be awarded to communities that use science-based prevention programs. 

Also, there is increasing recognition that informally or poorly run programs can do more harm than 
good in organizational or community settings. As M. A. Pentz points out (personal communication, 
April 2001), loss of program fidelity can sometimes lead to chaos, because once the program has been 
modified, no one quite knows how it will operate or what unexpected consequences it will produce. 
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On the other hand, prevention agencies and communities also face pressures for adaptation. To many 
observers, program fidelity represents a “top-down” approach to implementing a substance abuse pre- 
vention program, while adaptation represents a “bottom- up” approach that may have broader political 
appeal. Some communities may simply have an aversion to “copying;” they want to be “different,” to 
“be creative ” 

Even in the best-developed programs, there are often differences in community environments and tar- 
get populations that really do necessitate certain kinds of modifications. Many of these ultimately are 
about resource limits, as in the example about Life Skills Training previously described. If a teacher has 
only so many classroom hours to devote to a prevention program, either a longer program is modified, 
or it is not used at all. The issue, as previously noted, is one of boundaries: How much fidelity is essen- 
tial? How much adaptation is possible? 
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Review of the Literature on 
Stages of Program Implementation 

The stages of program implementation depicted in Exhibit 1 provide a conceptual framework for organ- 
izing the review of wide-ranging literature on the question: “How can the balance between program 
fidelity and adaptation best be set in order to promote successful implementation of science-based sub- 
stance abuse prevention programs?” 

Even though the seven stages noted above are not always followed precisely in the real world of pre- 
vention programming — sometimes stages are skipped, or minimized, or conducted simultaneously, or 
re-arranged — this paradigm presents a useful way of conceptualizing and evaluating whatever process 
does get followed. 

As noted earlier, there are activities prior to implementation (program development, validation, and dis- 
semination), as well as post-implementation (feedback and program modification). However, this liter- 
ature review focuses on the seven stages in the program implementation model. 

Program Adoption 

Implementation begins by using available information about a substance abuse prevention program to 
make a decision to adopt the program. Based on more than 80 years of empirical research (Backer, 
1991; Backer, David, & Soucy, 1995; Rogers, 1995b), four key factors weigh most heavily in deter- 
mining whether information disseminated to potential implementers will lead to adoption of a program: 

1. how information about the characteristics of the program itself is communicated to 
different key target audiences; 

2. how information about the efficacy of the program, including both main and side 
effects, is communicated to particular audiences; 

3. how the necessary financial and human resources to support the program’s imple- 
mentation are identified; and 

4. how the complex human dynamics of change are dealt with, such as addressing fears 
and resistances about implementing a program and building a feeling of ownership 
and participation in it. 
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Application of these science-based principles for disseminating and implementing programs to the sub- 
stance abuse prevention field is still limited (Backer, 2000; Backer, Brown & Howard, 1994; Brown, 
1998; Lamb, Greenlick, & McCarty 1998; Rohrbach, D’Onofrio, Backer, & Montgomery, 1996). Often 
the result is failure to adopt a science-based program that might, in fact, have considerable potential for 
addressing substance abuse prevention goals in a particular setting. 

Principles about how people and organizations deal with innovation and change also are reflected in 
research on fidelity/adaptation and other aspects of program implementation. For instance, Fullan and 
Pomfret (1977) assert that: 

If there is one finding that stands out in our review, it is that effective implementation 
of social innovations requires time, personal interaction and contacts, in-service train- 
ing and other forms of people-based support. Research has shown time and time again 
that there is no substitute for the primacy of personal contact among implementers, 
and between implementers and planners/consultants, if the difficult process of 
unlearning old roles and learning new ones is to occur. Equally clear is the absence 
of such opportunities on a regular basis during the planning and implementation of 
most innovations, (p. 391) 

Mayer and Davidson (2000) frame adoption (and later implementation) as part of a larger social change 
model. In this model, program, implementer, and environment interact in a unique pattern that must be 
addressed sensitively in order for successful change to occur. 

Gray, Emshoff, Jakes, and Blakely (under review) add the component of the research process to this 
social change approach, in the “Experimental Social Innovation and Dissemination” model. This model 
has been used in a number of discussions about dissemination of education in fields like school health 
(Basch, 1984). The Hall and Hord (2001) Concerns-Based Adoption Model (CBAM) approach (to be 
discussed in more detail below) also supports attention to the variables discussed here. 

Overall, the science on program adoption and program implementation (including fidelity/adaptation 
balance) all leads in the same direction: to the usefulness of strategies and principles like the ones out- 
lined in this section in promoting success for the complicated process of change. 
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Needs and Assets Assessment 

A community needs assessment often precedes program implementation. It uses informal processes 
such as group discussions, or a formal study through written surveys or structured interviews, to iden- 
tify community problems in substance abuse or related areas (such as violence) for various target pop- 
ulations (most often, the community’s youth). In recent years, this deficit approach has been comple- 
mented by an assets-based method pioneered by Kretzmann and McKnight (1993). This latter method 
looks for formal and informal community resources that can contribute positively to the implementa- 
tion of an innovation. 

Arthur and Blitz (2000) provide a paradigm for needs assessment and strategic community planning at 
the front end of program implementation, which can be used specifically with a focus on fidelity/adap- 
tation balance. This model requires both needs assessment and assets assessment. (The latter may use 
approaches such as the Search Institute’s “positive youth assets” and McKnight’s Asset-Based 
Community Development, or ABCD). Strategic community planning of this type involves not just the 
adopting organization but the entire community. 



Readiness Assessment 

As emphasized by Backer (1995a), needs and assets assessment are not the same as readiness assess- 
ment. Organizations or communities often need to implement a program and have the assets to do so, 
but if the key players are not ready to do so, it is unlikely that the program will succeed. 

Getting et al. (1995) emphasize that substance abuse prevention interventions often fail not because 
they are inadequate, but because communities are not ready to accept and implement them effectively. 
Therefore, assessing readiness and intervening to enhance community readiness for the kinds of 
changes required by a drug abuse prevention program are action steps that can greatly increase the actu- 
al effectiveness of substance abuse prevention programs. 

Getting et al. (1995) developed a nine-stage model of community readiness for substance abuse pre- 
vention programming. Edwards, Jumper-Thurman, Plested, Getting, and Swanson (2000) provide a 
practitioner-oriented description of the theory base behind this model, outline a key informant interview 
method for evaluating readiness, and offer strategies useful for enhancing readiness. 

Kumpfer, Whiteside, and Wandersman (1997) also have developed a model of readiness assessment and 
enhancement specific to the substance abuse prevention field. Backer (1995a) reviewed the larger lit- 
erature in behavioral and management sciences and presents a number of approaches to assessing and 
enhancing readiness for change from those areas which have implications for the drug abuse field. 
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Program Assessment 

Three types of program assessments may also be helpful: 

Theory and/or Logic Model 

Most programs were initially based on theoretical assumptions about what produces behavior change 
in their target populations (e.g., the underlying reasons teenagers do or do not use drugs, such as peer 
influences). Knowing what that theory is can help implementers successfully use the programs. 

Theory also can be specified in a logic model, which sets forth a chain of conditions for the interven- 
tions (also called “mediators”) and their presumed results. The logic model can be compared with actu- 
al data gathered at each step, helping to establish a chain of causation that may be difficult to detect 
from simply looking at the intervention made and the result achieved. Logic models have been much 
discussed in the recent program evaluation literature. Goodman (2000) calls for the use of logic mod- 
els in prevention, and cites them as prime approaches for assessing implementation fidelity (see also 
Goodman & Wandersman, 1994; Kumpfer et al., 1993; Scheirer, 1996). 

Sometimes program implementers actually stop at the level of theory. They implement only their under- 
standing of the theory behind a program rather than the program itself. This may have value in some 
circumstances, but it represents a drastic amount of program adaptation under the implementation 
stages approach suggested here. 

Sometimes logic models also are developed atheoretically, on the basis of straightforward operating 
components of a program. And sometimes, an overall theoretical construct affects every element of pro- 
gram implementation. For instance, the Child Development Project (Battistich, Schaps, Watson, 
Solomon, & Lewis 2000) is a comprehensive, ecological approach to school-based prevention (includ- 
ing, but not restricted to, substance abuse issues) in which proper implementation means that “the whole 
school becomes the program.” 

Core Components Analysis 

Methods for conducting core components analysis include analysis of program materials such as man- 
uals; actual observation of programs as they are implemented using some sort of rating approach; and 
secondary analysis of a program in published literature — or some combination of these methods. These 
methods can be used by program developers, independent scientists studying prevention programs, and 
program implementers. 

Most core components in a prevention program operate simultaneously or sequentially. There is an 
assumption that their effects are synergistic. However, research to test this assumption has yet to be 
undertaken, as M. A. Pentz (personal communication, April 2001) and others have pointed out. 
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The National Institute on Drug Abuse has initiated a new funding effort to support empirical research 
aimed at identifying the component parts of effective prevention programs. This initiative is discussed 
further below. 

These methods have been discussed in the program evaluation literature, and also in the prevention lit- 
erature (e.g., Basch, 1984; Durlak, 1995). For instance, Cook, Leviton, and Shadish (1985) character- 
ize essential program elements as the components of a project necessary for effectiveness. Good pro- 
gram theory generates a causal model that links these components together. Dalton, Elias, and 
Wandersman (2000) advise developers to specify the key components of their programs, to dovetail 
with efforts made by program implementers. 

Core components analysis has been done in a number of other fields as well. In education, for exam- 
ple, Backer (1995b) describes core components analysis done by the Soros Foundation for the replica- 
tion of the American Head Start Program in Eastern Europe. Also in the education field, the Concems- 
Based Adoption Model (Hall & Hord, 2001) involves a core components analysis using the Innovation 
Configuration Map process. 

Backer (1999) describes a core components analysis for the Arts Marketing Collaborative initiative of 
the John S. and James L. Knight Foundation. McGrew and Bond (1995) describe core components 
analysis for Assertive Community Treatment, a well-vali dated mental health treatment model, using an 
empirical procedure that may also have implications for the substance abuse prevention field. 

As science-based substance abuse prevention programs have proliferated, the same independent analy- 
sis has been applied to the core components for a number of programs (Backer & Rogers, 1999). For 
example, Dusenbury and Falco (1995) identified 11 core components of school-based prevention pro- 
grams. Hawkins and Arthur (2001) describe the Seven State Diffusion Project and its core components 
analysis in the substance abuse prevention area. 

CSAP, through its National Center for the Advancement of Prevention, is conducting a large-scale core 
components analysis, as part of developing and maintaining the National Registry of Effective 
Prevention Programs (NREPP). As of early 2001, NREPP contains 39 programs, with another six “in 
the pipeline” of final review. 

This work started with analysis of fidelity instruments for the seven CSAP High Risk Youth model pro- 
grams. From this developed a method for identifying core components and rating their degree of imple- 
mentation. Data were sent to independent reviewers for appraisal and subsequent modification. All of 
the NREPP programs will be appraised using the revised method (S. Schinke, personal communication, 
October 2000. For more information about the CSAP core components analysis, contact the Center for 
Substance Abuse Prevention or consult its online database, www.samhsa.gov/csap/modelprograms/). 
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Core components analysis can lead directly to the development of fidelity instruments for measuring 
the implementation of prevention programs. However, as discussed below, such instruments are some- 
times developed less formally. 

Developer Consultation 

The final stage in this pre-implementation assessment involves interaction with the program developer 
to leam about problems and opportunities in program implementation. This may be especially impor- 
tant in obtaining information about program fidelity/adaptation balance. G. E. Hall (personal commu- 
nication, March 2001) emphasizes that the ideal situation is one in which two-way communication 
occurs between developer and implementers. This permits fine-tuning the process by which the pro- 
gram is put into place in a given setting. 



Program Implementation 

The actual implementation process for a science-based prevention program includes five sub-stages. 
This section addresses the literature on each: 

1. setting the balance between fidelity and adaptation; 

2. addressing other aspects of the implementation process itself, such as training for 
implementation; 

3. dealing with characteristics of the implementation setting, such as management sup- 
port from a school district, or the organizational climate within a community-based 
substance abuse prevention agency; 

4. dealing with characteristics of the clients being served by the program; and 

5. putting the program into the larger environmental context of the community, public 
attitudes about prevention, etc. 

These sub-stages also are discussed in detail by Chen (1998). Chen argues that while an intervention 
(the program) is the major change agent in any substance abuse prevention effort, the “implementation 
system,” as outlined above, is also likely to contribute to program outcomes in important ways. 

Fidelity/Adaptation Balance 

In a recent comprehensive analysis of fidelity/adaptation issues and the larger context of program 
implementation, Mayer and Davidson (2000) note that these concepts all emerged originally from the 
program evaluation field. As mentioned earlier, they sprang from efforts to explain why so many eval- 
uation results with previously validated programs were null or inconsistent. 
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Research, practice, and theoretical discussions about fidelity/adaptation balance are colored by the com- 
plex human dynamics of cultural differences. These include, for example, differences between aca- 
demically based program developers, who are often researchers as well, and community-based program 
implementers, who are sometimes grassroots advocates as well. 

Differences may also arise from competitiveness, (e.g., in academic publications; also, some science- 
based prevention programs are now sold commercially). Differences may arise, too, from communica- 
tion difficulties among people who come from these varied backgrounds. These differences are some- 
times further complicated by the number of subject fields in which fidelity/ adaptation issues have been 
discussed, each of which has its own nomenclature. 

Particularly the older literature on the subject speaks of investigators of fidelity and adaptation as sep- 
arated into two “warring camps.” One example is the debate in the literature between Fairweather, 
Tomatzky, and colleagues (e.g., Calsyn, Tomatzky, & Dittmar, 1977) and Backer, Glaser, and col- 
leagues (e.g., Glaser & Backer, 1977) about whether program adaptation does or does not contribute to 
sustainability (see section on sustainability below). 

Some of the distinctions are somewhat artificial; they are not to be encouraged (e.g., see Bauman, Stein, 
& Ireys, 1991). Certainly there is a vested interest in the “purity” of a program model by its creator, and 
a vested interest in local control by organizations and communities that are implementing the model 
programs. But increasingly, both “sides” recognize that the key to program success (as defined by both 
parties) is to strike a balance between fidelity and adaptation. The remaining problems of human 
dynamics are likely a matter of communication and coordination. 

The proponents of fidelity have tended to assume that altering a program in any significant way will 
reduce its effectiveness (Calsyn et al., 1977) and makes evaluation of the program across settings dif- 
ficult (Boruch & Gomez, 1977). However, many studies over the years have shown that programs sim- 
ply are not implemented with full fidelity no matter what exhortations or claims have been made by 
program developers. For instance, Tricker and Davis (1988), in a study of implementation of the Here’s 
Looking at You II and Starting Early curricula, found that teachers in one of the three school districts 
they studied implemented only half of the total lessons in HLAY II because of insufficient time with the 
materials. There are many other examples, as summarized in reviews like Mayer and Davidson (2000) 
and Domitrovich and Greenberg (2000). 

As previously noted, adaptation is also called “reinvention” in much of the literature on this subject, a 
term coined by Everett Rogers and his co-workers. Eveland, Rogers, and Klepper (1977) and Rogers 
(1995b) say that significant reinvention of programs is necessary to preserve program effectiveness. 
Rogers and colleagues have conducted research on this topic for more than 25 years. They found that 
programs across a broad range of topical areas and populations are reinvented — with the extent of 
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reinvention likely to increase over time — often because circumstances change and a program will not 
remain effective unless it is modified. 

In what is probably the largest study of educational innovations ever conducted, RAND Corporation 
researchers Berman and McLaughlin (1978) found that it was rare that an innovation was adopted pre- 
cisely. In most cases, not only was the innovation adapted, but changes occurred in the implementation 
setting as well, a process they term “mutual adaptation,” as already mentioned. In looking at a large 
number of educational innovations, mutual adaptation was the norm for the successful implementa- 
tions. However, as Datta (1981) has pointed out, innovations in this study were structured to encourage 
adaptation, which may have affected the outcomes observed. 

In a relatively early retrospective study of educational innovations (some criminal justice innovations 
are included as well), Blakely et al. (1987) reported that innovations implemented with fidelity had 
greater impact. However, they also found that implementation sites that added a program component 
were usually more effective than those that made no changes. This is what Palumbo, Maynard-Moody, 
and Wright (1984) term “constructive adaptation.” 

In Botvin et al. (I995)’s study of Life Skills Training, only an average of 60 percent of the program was 
implemented, and this result was felt to represent successful implementation. Adaptation, thus, was 
occurring on a large scale, based on observer ratings taken for this research. 

In this large, 56-school study, implementation fidelity was assessed by direct observation of randomly 
selected classes, using quantitative data-gathering measures. The comparison of outcomes made 
between the total sample of more than 3,000 students and those who were in high-fidelity classrooms 
showed small differences in favor of the high-fidelity implementations. The researchers concluded, 
however, that in public health terms, these small differences could have a substantial impact on mor- 
tality decreases from preventing smoking, if applied on a national level. 

An earlier study (Botvin et al., 1989), focusing on urban black youth, found that (despite reasonably good 
fidelity of implementation of the program) improving the program’s effectiveness may require more atten- 
tion to selecting high quality teachers and on training and supporting the teachers to ensure a high degree 
of fidelity. As other studies have found, program fidelity is neither automatic nor easy to achieve. 

Emshoff et al. (under review) report the results of a fidelity/adaptation analysis of the CSAP model 
replication grant initiative, which funded replications of 11 science-based prevention programs, each 
with its own instrument for measuring fidelity (described further below). This study suggested that 
higher fidelity replications were generally more successful. However, there were a number of excep- 
tions. For instance, program fidelity tended to be less important when there was lower cultural and envi- 
ronmental similarity between the original site and the adopter site. 
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In this research, mean fidelity scores fell between 1.47 to 1.88 (out of a possible 2 standing for ideal). 
Thus, there was considerable, but not complete, fidelity of implementation. The investigators reported 
that a lot of pressure for fidelity came from the funder to the grantees, which may have inflated the 
results. However, implementers consistently reported that they were happy to have had this pressure, 
since it led to better program outcomes over time. 

Finding the ideal balance between fidelity and adaptation was a deliberate process, the investigators 
reported. Some implementers asserted that it was most important to maintain rigorously the principles 
of the program, not specific mechanical procedures. This brings into question just what was being held 
to so closely, because this statement leaves the door open for much adaptation to have taken place. 
However, the overall result was impact fairly close to that of the original implementation for about half 
of the replications. 

Emshoff et al. conclude that “The conflict between fidelity and adaptation needs to be re-framed as a 
balance between the two processes. Considerably more research needs to be done to better understand 
the contexts that influence the ideal balance points.” Emshoff et al. also abstract from their experiences 
with the CSAP replication program what could be generalized as a two-part “model” intervention to 
promote fidelity: 

1. funding support: setting up funding so that fidelity adherence is required as a condi- 
tion of funding, which helps to motivate grantees and increase their attention to fideli- 
ty issues, as already observed; 

2. technical assistance: technical assistance from the program developer needs to be 
paid for by a funder, because the developers usually cannot pay for the time and 
expenses involved. 

In the case of the CSAP replication program, technical assistance included site visits to developer sites, 
site visits from developers, and a “replication manual” for each program, which detailed how to imple- 
ment the program in a new setting. CSAP staff also provided some technical assistance. Just having 
access to the fidelity instrument was useful for developmental purposes, according to personnel in the 
replication sites. 



Flay et al. (1987), in a quasi-experimental trial of a television and school-based smoking prevention 
program, found significant problems in the fidelity of implementation for this program. They suggest- 
ed that before such programs are disseminated widely, research needs to be done to determine what 
'level of fidelity is needed in order to achieve significant program effects (e.g., to deter smoking or drug 
use onset behaviors). 

For more than 25 years. Gene Hall and his associates have been conducting research and developing an 
underlying theory of educational innovation and change, referred to as the “Concems-Based Adoption 
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Model” (CBAM) (Hall, Loucks, Rutherford, & Newlove, 1975; Hall, 1992; Hall & Hord, 2001). Unlike 
most of the other work cited here, CBAM is not a specific program, but rather a strategy for promoting 
effective implementation. It guides change through a “change facilitator team,” assuming that both 
users and non-users of the innovation may result. Both exist in a “user system culture” that has an 
impact on the outcome of the overall effort to implement change. 

CBAM has three critical components, each of which is highly relevant to this discussion of 
fidelity /adaptation balance: 

• Innovation Configuration: This element of the CBAM model is concerned with 
defining the core and ancillary characteristics of an innovation so that its level of use 
can be determined. A complex “Innovation Configuration Map” is then built, with 
precise descriptions of each component and what constitutes its proper or improper 
use. Innovation configuration is related to some aspects of both fidelity/adaptation 
and core components analysis as defined here. 

• Stages of Concern: This is a six-stage conceptual model for understanding people’s 
feelings and perceptions about the innovation and the change required for its imple- 
mentation, ranging from basic awareness of what the innovation is, to a level involv- 
ing ways to improve the innovation after its successful implementation. A Stages of 
Concern Questionnaire, SoCQ, has been designed to measure these feeling reactions 
in potential adopters of innovations. This CBAM model component is most closely 
related to readiness, as discussed above. 

• Levels of Use: This is an eight- factor scheme for characterizing people’s behaviors in 
terms of what happens to an innovation that is being promoted for adoption, ranging 
from non-use, through integration and renewal. This again is about modifying the 
innovation to improve it after it is in stable usage. An LOU scale has been designed 
to determine where innovations fall as a consequence of the interventions. 

In the CBAM approach, interventions can be made based upon the results of these three analyses, each 
of which increases the chance of implementation success overall. If done properly. Hall and colleagues 
argue, the chances for an appropriate balance between needed fidelity to the program and needed adap- 
tation are also increased. 

It is significant to note that each of the key CBAM model components begs the question of program 
adaptation. A natural part of the life cycle of innovations, in terms of people’s feelings about them (SoC) 
and their actual use (LOU), involves changes in the innovation. This raises the important notion that 
fidelity/adaptation balance is a living variable, which may change during the life cycle of an innovation. 

A study of Resistance Training and Normative Education, two prevention programs studied as part of 
the Hansen Adolescent Alcohol Prevention Trial, focused on “program integrity,” their term for 
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fidelity (Hansen, Graham, Wolkenstein, & Rohrbach, 1991). Program integrity was measured by the 
program specialists who taught the program to students and by trained observers. 

Ratings of program integrity were significantly related to outcomes for three of seven mediating vari- 
ables studied. This suggests that fidelity may play an important moderating function on prevention pro- 
gram effectiveness, especially in immediate outcomes for a resistance skills training program. When the 
program was well taught, students’ resistance skills increased dramatically. The researchers created a 
general index of fidelity, which was used both by participants and observers to rate the quality of pro- 
gram delivery. 

In studying the All Stars Community Program, Hansen (1999) used state supervisor ratings of program 
fidelity. The results with this rather crude measure, used as part of a pilot program, suggest that pro- 
grams implemented with high fidelity had better overall effectiveness ratings. 

In an effort to understand some of the reasons for variations in fidelity of program implementation, 
Hansen and McNeal (1999) studied drug education practices among teachers in 12 middle schools, 
using a minute-by-minute observational approach. They found that teachers understood concepts other 
than knowledge transmission poorly, and this led to their ignoring other approaches that address risk 
and protective factors and have been shown by research to have programmatic effectiveness in delay- 
ing drug use onset. They conclude that “a relatively radical transformation of approaches to teaching 
will be needed” if drug education is to be as successful as research has shown it can be. 

Pentz et al. (1990), in one of the many studies reported from the Midwest Prevention Project (MPP), 
also called Project STAR, discuss the effects of program implementation on adolescent drug use behav- 
ior. Trained teachers implemented MPP, a school and community-based program for drug abuse pre- 
vention. Implementation was measured by teacher self-report and validated by research staff reports. A 
10-step model for community organization was created to help support MPP’s implementation in the 
community, including specific efforts to reinforce the implementers for their work. 

Quality of implementation (program fidelity) was operationalized in this study to include: 

a. adherence (the program was implemented in experimental groups but not in control 
groups); 

b. exposure (the amount of a program delivered to the target audience); and 

c. reinvention (the extent to which implementation deviates from the program standard). 

It found that a high level of implementation can produce actual declines or prevent 
increases in drug use prevalence rates. However, some reinvention is acceptable, and 
even desirable. This was operationalized as giving latitude to teachers to personalize 
the program to fit their students (Pentz et al., 1990). 
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Sobol et al. (1989) conducted a study of the integrity of delivery of the resistance skills training com- 
ponent of a social resistance smoking prevention curriculum. An “integrity index” was computed for 
the teachers delivering the curriculum, including their degree of adherence of the program and the qual- 
ity of their delivery. 

Teachers who scored above the median on this index were also more animated, articulate, and confi- 
dent during program delivery and less authoritarian in their style. The study was notable in that it was 
one of the first to use ratings (of videotapes made of the teachers at work) by independent judges to 
measure fidelity. This is seen as an* improvement over the typical self-report methodology. 

However, this study also points up a possible confounding variable common to many of the studies 
reported in this section. It is possible that the results observed come not from the teachers’ more faith- 
ful implementation of the program, but simply that they are better, more conscientious teachers. 

Spoth, Redmond, and Shin (1998), in a study involving implementation of Preparing for the Drug-Free 
Years (PDFY), promoted program fidelity by putting essential program content on videotapes. This was 
to ensure its standardized delivery and to enhance the learning process by visually demonstrating com- 
petent parent-child and family interactions. Trained observers monitored the fidelity of implementation 
of the intervention, with a focus on coverage of the program content by group leaders. Results showed 
that 69 percent of the small-component tasks were covered by the leaders. 

In this same study, implementation of the Iowa Strengthening Families Program (ISFP) was monitored 
for fidelity, showing that 83 to 89 percent of the component tasks described in the group leaders man- 
ual were covered in the family, parent, and youth sessions for this program. 

Spoth, Redmond, and Lepper (1999) examined dosage-related effects of the Iowa Strengthening Families 
Program (ISFP), as recommended by Botvin et al. (1995). Outcomes were compared between children who 
attended at least four of the seven sessions versus the entire sample. Reduction rates in alcohol use were 
higher for these “high-dosage children” (i.e., recipients of a high-fidelity program intervention) than for the 
entire sample, but this effect did not persist when two-year follow-up data were examined. 

These matters of fidelity and adaptation have also been explored in a number of areas outside substance 
abuse prevention, as summarized by Domitrovich and Greenberg (2000) in a recent analytic review. 
Without attempting a comprehensive assessment of these related works, a few from the health and edu- 
cation fields are mentioned here for comparative purposes. 

Allen, Philliber, and Hoggson (1990) conducted a national replication of a school-based teen pregnan- 
cy prevention program, using a fidelity measure of program intensity, program structure, and curricu- 
lum use. They found mixed support for fidelity. Only one of three measures of fidelity was related to 
outcomes: program intensity was; program structure and use of specific program modules were not. 
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Connell, Turner, and Mason (1985) found that in implementing a school health education program, a 
higher level of fidelity in program implementation produced only somewhat greater outcomes overall, 
but very strong differences in improved attitudes about health and self-reported health behaviors. 
Fidelity was measured in terms of 

1. instructional hours at or greater than minimums established by program designers, 

2. teaching of 80 percent or more of program activities, and 

3. greater than average degree of fidelity to the program materials used. 

Dane and Schneider (1998) examined the extent to which five aspects of fidelity were verified in eval- 
uations of primary and secondary prevention programs; 

1. degree to which program components were delivered as prescribed (what they call 
adherence); 

2. frequency and duration of the program administered (dosage); 

3. qualitative aspects of the program delivery (e.g., content, affective quality); 

4. participant responsiveness to the program; and 

5. program differentiation (efforts to verify that design conditions apply only to experi- 
mental groups receiving an intervention). 

Only 39 of 162 outcome studies featured specific procedures for documenting fidelity. And of these, only 
13 considered variations in integrity when analyzing the effects of the program. There was mixed evidence 
about the impact of fidelity levels on program outcomes in the studies for which such data were available. 

Detrich (1999) reported a study of programs for serving children with autism, finding that fidelity levels 
can be increased by matching intervention procedures to contextual variables in the classroom. That is, 
when adaptations were made that matched the program to the types of interactions teachers were observed 
to have with their students, the teachers were more likely to follow the adapted program faithfully. 

Domitrovich and Greenberg (2000), in addition to appraising progress in the subject fields mentioned 
at the beginning of this paper, also conducted an actual study of fidelity issues related to 34 science- 
based programs that prevent mental disorder in school-aged children. They used the definition of fideli- 
ty advanced by Dane and Schneider (1998) for this purpose, finding that 76 percent of these programs 
verified program integrity in some way. They also discussed the conceptual challenges of program 
adaptation, though their study did not measure adaptations made by the 34 programs. 

George, Hall, and Uchiyama (2000) report on the implementation of a standards-based approach to 
teaching mathematics, finding that student achievement outcomes are significantly associated with 
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higher teacher fidelity of implementation. They also found that, as measured by the Hall and associates* 
Levels of Use instrument (Hall et al., 1975), implementation of this major change in classroom prac- 
tices took place slowly, over several years. 

Parcel et al. (1991) studied ways to enhance the implementation of the Teenage Health Teaching 
Modules, finding that to improve fidelity among new THTM teachers, it is important that they be appro- 
priately trained and that they complete an implementation plan before using the curriculum. It also is use- 
ful to include teachers in the initial process of adopting the new curriculum. Even teachers experienced with 
THTM should receive continuing education to remain proficient in the program’s implementation. 

Parcel and colleagues measured both fidelity (lack of curriculum modification plus high percentage of 
required activities taught) and proficiency (progression in adaptation of the curriculum to meet student 
needs). They found that both were related to higher student knowledge gains among teachers new to 
THTM. However, for teachers more experienced with THTM, only proficiency was associated with 
student knowledge gain. These results may indicate that once teachers became experienced with using 
the curriculum, they could make modifications and still have an impact on student knowledge levels, 
as long as the teachers remained proficient in using the curriculum. 

Implementation Process 

Implementation was long thought to be an event in the life of an innovative program that would hap- 
pen relatively automatically if the innovation was of good quality and information was made available 
(Rogers, 1995b; Backer, 1991; Backer, David, & Soucy, 1995), More recently, implementation is seen 
as a complex process that can be thwarted at many levels (e.g., by program developers who do not think 
it is their concern, by funders who do not require attention to it, or by implementers of programs who 
do not pay proper attention to it). 

Providing training for the prevention workers, and for others whose behavior will have to change to 
make the program work in its new setting, is at the heart of the implementation process for substance 
abuse prevention programs. And their behavior has to change in such a way as to permit routine deliv- 
ery of the new program. Constraints which can negatively affect the implementation process include: 

a. lack of time to train implementers, 

b. lack of personnel to serve in implementing roles, 

c. lack of funds for training materials or for release time to permit training to occur, and 

d. lack of administrative support to promote a program’s smooth operation. 
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The research studies reviewed below describe other elements of the implementation process. Most of 
this research deals with successful implementations. However, there are also many failed implementa- 
tions. Often implementation failures are unobserved and, even more often, unmeasured. Thus, we can- 
not learn from them. Moreover, too often the successes are taken as discrete events, rather than process- 
es. The approach suggested here requires looking at implementation as a multi-stage event. 

Domitrovich and Greenberg (2000), whose work in this area was mentioned above, advance five ratio- 
nales for the importance of conducting systematic research on the implementation process: 

1. to know what actually happened during the implementation of a program, (e.g., to 
use in explaining program outcomes) 

2. to establish the internal validity of a program (e.g., addressing the dangers of a “type 
III error,” such as assuming that negative outcomes mean that the program is poor, 
when, in fact, they mean that its implementation was poor); 

3. to understand the internal dynamics and operations of a program and its overall 
strengths and weaknesses; 

4. to provide feedback for continuous quality improvement of the program; 

5. to advance knowledge about effective strategies for implementation. 

As the literature review that follows shows, despite these important goals, the study of implementation 
process has been limited to date. For instance, Durlak (1997) noted that less than 5 percent of more than 
1200 published prevention studies provide data on program implementation. Durlak and Wells’s (1998) 
meta-analysis of prevention programs found that 68.5 percent were described too broadly to be repli- 
cated and few included measurement of fidelity. Gresham, Gansle, Noell, Cohen, and Rosenbaum 
(1993), in reviewing school-based prevention programs, found that only 14.9 percent measured imple- 
mentation integrity. Furthermore, as Dane and Schneider (1998) point out, “promoting” integrity (e.g., 
use of a manual, providing training, etc.) is not the same as “verifying” integrity (gathering research 
data that these interventions were effective in high-quality implementation). 

Following are some key studies from the literature on the process of program implementation: 

Bauman et al. (1991), previously cited in the fidelity/adaptation section, discuss the importance of site 
characteristics, social and political context, and population characteristics. These program characteris- 
tics interact to produce unique circumstances affecting implementation. 

Berman (1980) suggests two different kinds of implementation: programmed implementation (highly 
planned and routinized in advance) and adaptive implementation (in response to local circumstances). 
A contingency analysis allows good decisionmaking about which is appropriate to a given situation. 
Berman offers the following analytic chart to assist with this analysis: 
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Situation Type 



Situational Parameters 


Structured 


Unstructured 


scope of change 


incremental 


major 


certainty of technology or theory 


certain within risk 


uncertain 


conflict over policy goals and means 


low conflict 


high conflict 


structure of institutional setting 


stable 


unstable 



He suggests that this analysis be undertaken, with the larger perspective that structured situations work 
better with programmed implementation, and unstructured work better with process-oriented imple- 
mentation, as common sense would suggest. 

Berman and McLaughlin’s (1978) studies (cited earlier under fidelity/adaptation balance) also found 
that projects demanding little change in teacher behavior were likely to be implemented in a structured 
way; and ambitious change efforts that engaged the professionalism of teachers could be made to work 
with a more adaptive approach. The critical misconception is that the tightness of programmed imple- 
mentation can compensate for a loosely coupled structure. Many situations call for a combination of the 
two approaches. Stability of environment is also an important consideration. 

Berman (1981) also criticizes the old technological experimental paradigm in which innovations are 
fixed and have to be implemented like scientific experiments. He instead proposes a new paradigm: 

1. change occurs as a result of an implementation dominant process, in which imple- 
menting innovation generally requires changes; 

2. change requires three processes — mobilization, implementation, and institutionaliza- 
tion — which are loosely coupled, and not linear; 

3. outcomes of change efforts are time dependent and context dependent. 

There are always issues of clarification. People have to understand the change with some clarity, and 
this often evolves during the implementation process. 

Programmed implementation is highly planned and routinized in advance, while adaptive implementa- 
tion is in response to local circumstances. Contingency analysis lets decisionmakers evaluate which is 
appropriate to a given situation. 
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Dalton, Elias, and Wandersman (2000) provide a community psychology perspective on program 
implementation. They begin with the assumption that prevention innovations are complex operator- 
dependent innovations. Implementing them successfully is difficult because they are greatly influenced 
by decisions made by the people who carry them out. They are also context dependent, fragile in their 
basic structure, and difficult to specify. They must interact with a number of elements within an organ- 
ization or community to be effective. As a result, many innovative programs, even those supported by 
good science, do not get widely implemented, and they cite several studies to show this is the case. They 
offer a “conductor’s guide” to orchestrating implementation of prevention programs. 

Felner, Phillips, DuBois, and Lease (1991) review a number of studies that have defined and measured 
implementation, both from a fidelity perspective and an adaptation or process perspective. In addition to 
discussing conceptual models for evaluation and how these fit with implementation and fidelity/adaptation 
concerns, the authors identify a number of measures used in such research. These include: observation tech- 
niques, focused interviews, questionnaires, and content analysis for the program or its key documents. 

Felner and associates discuss the importance of understanding program implementation before evalu- 
ating program effects. They focus on the need to understand the relationship between the fidelity of pro- 
gram implementation, program dosage intensity, and the impact of program effectiveness. They con- 
clude that critical distinctions often fail to be made between programs as designed, programs as imple- 
mented, and the impact of such divergence on outcomes. 

In an early analysis of the literature on this subject, Fullan and Pomfret (1977) identified five compo- 
nents of innovative program implementation in education, based on 15 case studies in which attempts 
were made to measure implementation. The five components include: 

1. changes in materials (the program content) 

2. changes in structure (the system by which the program is delivered) 

3. changes in role/behavior of the people who implement the program 

4. changes in knowledge and understanding of these people 

5. program value internalization by the program implementers 

The measurement of implementation is a snapshot of what is happening to the innovation at a given 
point in time, and different measures may be needed when examining innovations at different points in 
their life cycles, as argued earlier in this paper. 

Fullan and Pomfret (1977) also discuss the RAND studies of Berman and McLaughlin (1978), calling 
the measures of implementation used in this research “very weak.” They are based only on self-reports 
by users, which can be misleading; and they are global rather than specific. 
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Fullan and Pomfret assert that a strong emphasis on program fidelity (including measurement of fideli- 
ty with quantitative instruments) may be most applicable when studying implementation of prepack- 
aged, relatively explicit innovations. This approach is more questionable when studying innovations at 
earlier stages of development and use, where adaptation may be more likely. 



They conclude that most successful implementation research designs incorporate more than one kind 
of measurement approach. A comprehensive measurement strategy can encompass a wide range of vari- 
ables, such as the following: 



A. Characteristics of the Innovation 


C. Characteristics of the Adopting Unit 


1. Explicitness (what, who, when, how?) 


1. Adoption process 


2. Complexity 


2. Organizational climate 




3. Environmental support 


B. Implementation Strategies 


4. Demographic factors 


1 . In-service training 




2. Resource support (time and materials) 


D. Characteristics of Macro Sociopolitical Units 


3. Feedback mechanisms 


1 . Design questions 


4. Participation 


2. Incentive systems 




3. Evaluation 




4. Political complexity 



Some of the implications of this analysis for policymakers include: 



a. suggesting that the emphasis should be on both broad-based implementation of spe- 
cific models and 

b. local experimentation should be encouraged during implementation, with documen- 
tation of its results; and 

c. evaluation at least for initial implementation should be directed towards that process 
rather than outcomes. Fullan and Pomfret also conclude that the reward system for 
implementation efforts must be significantly revised if these aims are to be achieved. 

Gager and Elias (1997) tested a “resiliency model” for helping prevention programs to survive in high- 
risk environments, using interventions focused on resiliency factors similar to those found in children 
who thrive in high-risk environments. Variables predictive of program success can be derived from 
understanding the school system’s own implementation practices. These are not specific to a program, 
but are part of the school’s culture in embracing new programs. Their results show that prevention pro- 
grams can be successful even in high-risk school districts. Much of that success, however, relates to 
implementation practices, such as linking programs to the stated goals or missions of the school or dis- 
trict, and having programs carefully planned and carried out by well-trained personnel using engaging 
materials and procedures. 
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Goodman (2000) makes the fundamental point that implementation activities have shifted from organ- 
izations to communities, and this is another reason why encouraging certain kinds of local adaptations 
has become so critical. Goodman and his colleagues have looked at the role of community coalitions in 
developing community- focused implementations, and Goodman, McLeroy, Steckler, and Hoyle (1993) 
created a measuring technology for their institutionalization. They use a social ecology approach to 
understand the complex sociopolitical environment in which community decisions about implementa- 
tion are made and put to work at five levels: individual, interpersonal, organizational, community, and 
macro-policy. 

Leithwood and Montgomery (1980) present a methodology for evaluating implementation. It includes 
procedures for 

1. identifying descriptive dimensions of the innovation, 

2. specifying practices implied by the innovation, 

3. describing actual practices, and 

4. comparing actual with intended practices. 

McCormick, Steckler, and McLeroy (1995) studied implementation of school-based tobacco prevention 
curricula in 22 school districts, using three measures to assess implementation of the curricula. Larger 
organizational size and teacher training were the strongest predictors of successful implementation. A 
favorable organizational climate within the school district also favored implementation. Their measures 
were a Hall et al. (1975) Level of Use questionnaire and two administrations of an implementation 
checklist at two different times. 

Roberts-Gray and Gray (1983) present a model for implementing innovations based on Lewin’s (1951) 
theory of social change, beginning with orientation of the potential user, initiating the innovation, and 
ending with integrating the innovation into standard practice. These stages fit with the readiness, imple- 
mentation, and sustainability concepts discussed in this review. 

Scheirer and Rezmovic (1983) reviewed 75 projects that involved measuring implementation in nine 
content areas. More than half of these studies were conducted after 1979, the researchers observe, so 
this is a relatively young field. The personal interview was the primary data collection measure. They 
concluded that examinations of the relationship between program implementation and outcome are rare. 

Tomatzky and Klein (1982), in a meta- analysis of 75 studies, focused on the nature of the innovation 
itself in their research on implementation. They found, for instance, that three innovation characteris- 
tics (compatibility, relative advantage, and complexity) had the most significant relationship to whether 
an innovation was actually implemented. 
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Issues that have yet to be addressed fully in the literature on implementation include: 

7. Impact of Policy. Mayer and Davidson (2000), for instance, distinguish between “micro-implemen- 
tation” as discussed by Berman (1980), and policy or macro-implementation. Most of the studies of 
implementation concern micro-implementation in single organizations or communities, rather than 
waves of implementing programs based upon a funding initiative or public policy decision. The char- 
acteristics of the implementation process (including fidelity and adaptation issues) may be very differ- 
ent in such large-scale change. 

2. Efficacy of Alternative Methods for Implementation. Basen-Engquist et al. (1994) describe a com- 
parative study of live training workshop versus video training strategies for implementing Smart 
Choices, a school-based tobacco prevention curriculum. Their research found that a lower proportion 
of video-trained teachers implemented the curriculum, but that overall completeness and fidelity of 
implementation for those teachers who did teach the curriculum were comparable for the two groups. 
Video-trained teachers, however, were less likely to use brainstorming and student presentation/role 
plays, two of the methods prescribed by the curriculum, resulting in some impact on fidelity. These 
findings suggest that the implementation method needs to be considered carefully in terms of potential 
impact on fidelity and on overall level of implementation. 

Gingiss (1992) provides a number of strategies for enhancing program implementation in the school- 
based health education arena, all using a peer-based staff development approach. These include: 

a. strategies that are sensitive to the “developmental stages” most teachers pass through 
in learning to implement an innovation; 

b. peer collaboration in learning how to implement the innovation and in maintaining it 
over time; and 

c. putting these strategies into operation in the larger organizational context of staff 
development in a public school setting, ranging from encouraging risk-taking by 
teachers, to strong emphasis on implementation by school administrators. These 
methods may also have application in substance abuse prevention. 

3. Impact of Implementation Tactics on Program Validity. Palumbo and Oliverio (1989) provide argu- 
ments about how lack of attention to implementation can threaten the basic validity of an innovation 
and the measurement of that validity. Specific tactics of implementation may help protect program 
validity. For instance, when a program is in its initial stages of development, it necessarily will have 
somewhat vague goals and less precise connection with theory. Implementation tactics need to be 
accordingly flexible and adaptive in such situations. 
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4, Cautions about Implementation Strategy, Based on the literature reviewed here, the following five 
cautions need to be kept in mind when applying the knowledge base on program implementation, either 
at the policy or practice levels: 

a. If the bar is set too high for statistical evidence of program success, it may be possi- 
ble to have a successful program no one can use, so it will never be implemented. 

b. Communities or organizations must have the underlying capacity to participate in 
implementation (management skill, technology infrastructure, financial resources, etc.) 

If they do not, some type of capacity building experience must be provided to them. 

(Backer and Bare, 2000, discuss the growth of nonprofit capacity building services avail- 
able in many parts of the United States, which might be harnessed to this task). 

c. Durlak (1998) asserts that effective program implementation requires looking at the 
universe of potential participants to determine whether a significant number of them, in 
fact, are participating in the program (see section on client characteristics that follows). 

d. Durlak (1998) also presents this research concern. There are often a number of pre- 
vention programs operating simultaneously in an organizational or community envi- 
ronment. As a result, it may be nearly impossible to obtain a true “no treatment con- 
trol group” for the target population of interest. (He cites one case in which 90 per- 
cent of the students in a control group had, in fact, received another type of substance 
abuse prevention program during the period of the study.) 

e. There is considerable “wisdom literature” on program implementation — qualitative 
rather than quantitative in nature — which nonetheless provides valuable insights about 
how to implement a program with an appropriate balance between fidelity and adapta- 
tion. Some recent compilations of effective programs, such as Elliott (1998), contain 
sections presenting implementation concerns and practices based on actual experiences. 

Implementation Setting 

The organizational or community setting in which the program is implemented can greatly impact both 
the overall implementation process and fidelity/adaptation balance. Thus, community or organization- 
al interventions may be needed that are separate from program implementation per se (e.g., to remove 
general attitudinal or practical barriers that otherwise would compromise the success of the program). 

Hallfors, Sporer, Pankratz, and Godette (2000) conducted a survey of school personnel involved in the 
Safe and Drug Free Schools Program of the U.S. Department of Education. They found that limits in 
time and money were common barriers to full implementation of the program’s “Principles of 
Prevention,” (which include attention to fidelity). Many respondents were not well informed about sci- 
ence-based prevention programs, and reported that school districts often adapt the selected programs 
because of resource constraints in their particular environments. As L. McDonald (personal communi- 
cation, December 2000) points out, this kind of adaptation is often unhelpful. (She refers to it as “pro- 
gram drift,” which she says most often occurs because of financial issues). 
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Client Characteristics 

The psychological and demographic characteristics of a program’s clients also can influence imple- 
mentation or fidelity/adaptation balance. Some client characteristics may not greatly impact robuk pro- 
grams. For instance, many programs seem to be successful without modification of their basic compo- 
nents when used with various racial and ethnic subgroups (although there is anecdotal evidence, as well 
as long-standing belief, that careful attention must be paid to diversity issues in the implementation 
process). However, there is little question that some client characteristics can significantly impact the 
implementation process and its likely outcome (e.g., if clients are unusually difficult, resistant, etc.). 

Implementation as related to target population appears in a number of studies about use of prevention 
interventions originally validated with white (often middle-class) subjects, and then used with minori- 
ty (often poor) subjects (Backer & Diaz, 1999). The weight of evidence suggests that most substance 
abuse prevention programs that have been validated with one racial/ethnic group do not need to be sub- 
stantially modified to work effectively for another. However, language changes may need to be made 
(e.g., translation into Spanish). Backer and Diaz assert that without culturally modifying the imple- 
mentation strategy, resistance is likely within the adopting community, individually or collectively, 
even if the program’s scientifically tested efficacy does not require modification. 

The literature review in Spoth, Redmond, Kahn, and Shin (1997) shows that recruitment of families into 
preventive interventions is very challenging, particularly when an at-risk population is required. This is 
true even for the initial assessment stage of an intervention. In a prospective study of 1,121 families, an 
early statement of intention to participate and level of education were significant predictors of partici- 
pation in assessment and intervention. Obviously, if clients cannot be effectively recruited, implemen- 
tation will fail. Fidelity/adaptation balance might be tweaked to alleviate such situations. 

Attrition of participants from a prevention intervention can also be a problem. In a subsequent study, 
Spoth, Goldberg, and Redmond (1999) found that family socioeconomic status was the only variable 
that predicted attrition from the intervention under study. Program implementers may find that changes 
in a program’s fidelity/adaptation balance may help with attrition problems as well. 

Larger Context 

The larger social and political context for a program also can have an impact. Is there community sup- 
port? What is the overall place of substance abuse prevention on the public agenda? What other pre- 
vention programs or related community programs are in place? Everhart and Wanders man (2000), for 
instance, assert that feelings of community ownership and perceived capacity to deal with the issues of 
substance abuse prevention often have a disproportionate impact on whether a particular program will 
be implemented successfully. 
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Evaluation 



Evaluation in the field of implementation of substance abuse prevention programs mainly occurs in the 
following three arenas: 

a. measurement of program fidelity and adaptation, usually through use of a specialized 
fidelity instrument; 

b. measurement of the process by which the program is implemented: and 

c. outcomes in terms of impact on the prevention system and on particular target 
audiences. 

Fidelity Instruments 

At least since the work of Hall and associates (e.g., Hall et al., 1975), Berman and McLaughlin (1978) 
and Fullan and Pomfret (1977), alternative approaches have been available to measure empirically the 
level of fidelity of a program and the amount of adaptation that has taken place. Fullan and Pomfret 
identify four types of implementation measures, which can zero in on fidelity issues: observation tech- 
niques, focused interviews, questionnaires, and content analysis of key documents and curricula. They 
emphasize that until specific research establishes one of these methods as superior, it is best to use more 
than one measurement technique when studying fidelity of implementation. 

In a related field, Brekke and Wolkon (1988) report the development of a Daily Contact Log. This instru- 
ment is used to monitor fidelity of implementation of service programs for people with mental illnesses. 

Many science-based substance abuse prevention programs mentioned in this paper, such as Botvin’s 
Life Skills Training (Botvin et al., 1995) and McDonald’s Families and Schools Together (McDonald, 
1998) have well-developed fidelity instruments. CSAP is assembling a catalog of these instruments as 
part of the NREPP Core Components Analysis project mentioned above. 

Fidelity instruments were also constructed for the 1 1 CSAP model replication programs, as reported by 
Emshoff et al. (under review). Each fidelity instrument contains more than 100 items and was devel- 
oped in close collaboration with the program’s developers. In these instruments, each item is scored on 
a three-point scale, “ideal, acceptable, or unacceptable.” 

Process Evaluation 

Many evaluations of substance abuse prevention programs focus on the activities by which a program 
is implemented. Such data gathering can provide valuable input for program improvement as a program 
is being implemented. In addition, this is important documentation about what does and does not work 
in the process, which may be helpful to others using a program in the future. Good process evaluation 
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data also can help to build political and community support for a program and serve as an ongoing guide 
to the fidelity/adaptation process (Glaser & Backer, 1972; Fetterman, Wandersman, & Kaftarian, 1996). 

Outcome Evaluation 

Ultimately, those involved in funding, implementing, and using a program in an organizational or com- 
munity setting want to know whether it works. This requires outcome evaluation, which gathers evi- 
dence about the impact and cost-effectiveness of a program, such as reduction in substance abuse rates 
for a particular target population, increased awareness of the problem, etc. (Fetterman et al., 1996). 
Outcome evaluation also can provide evidence long-term about the quality of the implementation 
process and the balance between program fidelity and adaptation. 



Sustainability 

The sustainability of innovations is an issue that has been little explored in substance abuse prevention 
or other fields (Backer & Rogers, 1999). Sustainability means the long-term survival of a program, once 
it is successfully implemented (i.e., how to continue a program as long as nothing else better comes along 
to take its place and it continues to produce substantial positive results without major side effects). 

The existing empirical literature about what promotes or inhibits sustainability is based mostly on case 
study research. Some of the pertinent work includes: 

Akerlund (2000) provides a set of guidelines for program implementation that emphasize planning ahead 
for sustainability (e.g., looking toward long-term funding, ongoing community support, and refinements 
in operation). The author concludes that sustainable programs are the ones that remain adaptable, all 
other factors being equal. This analysis is in keeping with the bulk of the literature on this topic. 

Altman (1995) argues that sustaining interventions requires fostering long-term collaborations between 
researchers (or program developers) and communities. In addition, researchers need to look early on for 
ways that their research can benefit the community after the research is over, such as providing resources 
to translate the research study into an ongoing substance abuse prevention program that the community will 
value. 

Berman and McLaughlin (1978), in their previously cited study of educational innovations, found that 
long-term survival was relatively rare. Most of the educational innovations they studied continued in 
name only, or as pale imitations of the original program. 

Glaser and Backer (1977) argue that adaptation is one of the elements that facilitate program sustain- 
ability. In two early studies of long-sustained programs in the mental health field (Goal Attainment 
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Scaling and the Fairweather Hospital -Lodge Community Treatment Program), Glaser and Backer 
(1980; Backer & Glaser, 1979) developed case studies based on site visits. Results showed key factors 
in long-term sustainability included: 

a. adaptation of the innovation due to changing community circumstances or service 
needs, as previously argued; and 

b. availability— early in the life cycle — of technical assistance on enhancing long-term 
survival strategies. 

Goodman and Steckler (1989) describe “niche saturation”: institutionalization through getting into 
every possible part of a more complex organization, such as all classrooms in a school or all schools in 
a district. Goodman et al. (1993) developed a measure of institutionalization, with eight sub-scales, to 
ineasure these and other elements of sustainability. 

Pentz (2000) addresses issues of policy change related to sustainability. She cites a number of studies 
indicating that both planned and unplanned policy change have a positive impact on program longevi- 
ty. She considers two types of policy change: 

a. those related to implementation, such as raising funds, requiring standard implemen- 
tation, and creating a formal nonprofit organization to implement programs; and 

b. those related to regulations, such as enforcing monitoring of drug-free zones. 

Pentz’s review indicates that regulatory policies may show the most immediate effects in implementing 
substance abuse prevention programs, but programmatic policies are more likely to have long-term con- 
sequences. Barriers to sustainability include lack of perceived empowerment by community leaders to 
continue prevention work; insufficient preparation of community leaders for adoption of science-based 
programs; the temptation to continue an ineffective approach because of investments already made in 
it (as happened, for example, with the DARE program); and the general perception that no science- 
based approach will work because each community has its own unique needs. 

Tomatzky, Fergus, Avellar, and Fairweather (1980) studied both the impact of cycles of change (annu- 
al budget periods, for instance) and passages (fundamental shifts, such as legislation ending or dramat- 
ically changing a funding source) on mental health innovations within a sample of psychiatric hospi- 
tals. Overall, the results suggest that passages, much more than cycles, affect sustainability, presumably 
because the latter tend to involve smaller, more predictable changes. 

Yin (1978) studied the life history of six innovations in municipal bureaucracies. In each case, survival 
of an innovation was tracked through a series of passages (conversion from Federal to local funding 
support) and cycles (budget periods). 
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Revisiting Fidelity and Adaptation 

One factor cited in the above literature (e.g., Glaser & Backer, 1977) as significant in improving the 
chances for program sustainability is a planned, systematic revisiting of the balance between fidelity 
and adaptation as the program continues. Conununities and target populations change over time, and 
the program may need to be changed as a result. If there is either resistance to this, or lack of an infra- 
structure to make this revisiting process relatively straightforward, the long-term chances for survival 
are reduced. 

On the other hand, what L. McDonald (personal communication, December 2000) calls “program drift” 
can also occur (i.e., undesirable changes in a program of which the implementers may not even be 
aware). Revisiting a program can provide an opportunity to refresh it, to bring it back into line with its 
original intent and procedure. 

Routinization 

Perhaps the ultimate objective of efforts to promote sustainability of a substance abuse prevention pro- 
gram is institutionalization, or routinization. This means incorporation of the program into the ongoing 
life of the organization or community, so that sustainability becomes, in some respects, automatic. It 
might take the form of including funding support for the program in a permanent budget (e.g., as a line 
item in an annual city or county budget). Or, it might take the form of a policy for permanent continu- 
ation, by an appropriate governance body (city council, board of directors, etc.). No routinization is 
completely permanent, but these larger systems changes do increase the chances that a program will 
survive over time. 

Lastly, sometimes sustainability, even routinization, happens for the wrong reason! The weight of evi- 
dence suggests that DARE is not an effective substance abuse prevention program, yet it has had a 
remarkable degree of sustainability, even routinization, in American communities over the last decade. 
Rogers (1995a) outlines some of the reasons for this; for example, the DARE program has had extraor- 
dinary publicity, support from police chiefs (a powerful force in most communities), and good finan- 
cial backing. 
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Main Conclusions from 
the Literature Review 



This literature review started by conceptualizing seven stages of program implementation (Exhibit 1), 
including fidelity/adaptation balance as one sub-part in those stages. This conceptual framework has 
been used to organize the wide-ranging literature pertinent to the central question this paper addresses: 
How can the balance between program fidelity and adaptation best be set in order to promote effective 
implementation of science-based substance abuse prevention programs? 

The seven stages of implementation, as already stated, are not always followed precisely in the real 
world of prevention programming. Sometimes stages are skipped, or minimized, or conducted simulta- 
neously, or re-arranged. However, this paradigm represents a useful way of conceptualizing and evalu- 
ating whatever process is followed. 

The most important conclusion from the literature review presented here on each of these seven stages 
is that attention to BOTH fidelity and adaptation is essential for successful implementation of sci- 
ence>based substance abuse prevention programs. That is, fidelity/adaptation is not a continuum 
upon which each specific implementation of substance abuse prevention program falls. Rather, fideli- 
ty/adaptation balance is concerned with the complex, dynamic interaction between a program and its ^ 
environment. Science and experience say that maximum success requires attention to both fidelity and 
adaptation. 

Out of this assumption come guidelines for balancing fidelity and adaptation, which are presented 
below. These are based upon the literature reviewed and are presented as a preliminau’y response to 
requests from the prevention field for a practical means offsetting the boundaries” regarding this issue. 

Further refining and improving these guidelines also will require dealing with the issues for 
researchers, program developers, implementers, funders, and policymakers that follow. These repre- 
sent, in effect, additional conclusions from the literature review. The section on “Next Steps” offers 
some beginning points for addressing them. 

To date only limited attention has been paid to fidelity, adaptation, and other components of program 
implementation in both the science and practice of substance abuse prevention programming. Yet sci- 
ence shows that dealing with implementation issues is critical to program success. There is even a term 
in program evaluation — “type III error” — to label the significant number of cases where researchers 
conclude that a program is not effective, but the real problem is that it was not implemented properly. 
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Even with science-based programs and good implementation strategies (including attention to fideli- 
ty/adaptation balance), there is no guarantee that a program will lead to significantly improved out- 
comes, such as development of resistance skills, less use of drugs or alcohol than comparison groups, 
etc. Evaluating implementation efforts and measuring the balance set between fidelity and adaptation 
have meaning only in the context of outcomes, and these too must be measured by research. 

That is, most science-based substance abuse prevention programs now widely used in the field are 
research-based, but not research-verified (a concept well -discussed in the education field by George et 
al. (2000). They have been shown to work in an initial setting, but have not been verified through field 
research to prove they work in a wide range of organizational or community environments — and as 
implemented by people other than their developers. In many cases, even when effective implementation 
in fact happens, it is not verified by any sort of systematic data gathering (Dane & Schneider, 1998). 

As will be seen in the list of unresolved issues below, the research evidence about effective program 
implementation and about fidelity/adaptation balance is far from complete, and it is filled with contra- 
dictions and uncertainties. This literature review is one of the first attempts to draw together all of the 
relevant threads of inquiry so that judgments can be made about gaps in the research and potential 
strategies to follow at the several levels discussed below. 

There are still some basic issues of terminology as well as methodology to be resolved. Different inves- 
tigators have different definitions and terms for fidelity. The measures used by program developers to 
determine program fidelity vary in scope and precision. For instance, one major program is considered 
by the program developer to have “fidelity” if 60 percent of its sessions are administered, but this fig- 
ure is never mentioned in secondary discussions of the developer’s work by others, giving a potential- 
ly misleading impression. 

Despite these shortcomings and inconsistencies, however, the weight of evidence does suggest that the 
discussions that once divided program developers concerned with fidelity and program implementers 
concerned with adaptation into “warring camps” are now outdated. These discussions still happen both 
in the literature and in the field, but the view that fidelity/adaptation is a dynamic concept in which both 
elements are needed for program success makes these early arguments somewhat moot. Hall and Hord 
(2001) assert that asking the question “Is adaptation desirable?” is asking the wrong question. 
Adaptation will happen, so the questions to be asked instead are: “How much?” and “When is a pro- 
gram’s content damaged?” 

Bauman et al. (1991) refer to the “principle of program uniqueness” — that in its actual implementation, 
any program will have some unique elements because of the unique characteristics of the environment. 
Even if one is implementing a new program in exactly the same environment, the environment has 
changed since the time of the previous implementation! They also point out that many programs are 
created under unusual conditions (special funding, charismatic leaders, etc.) that are not widely avail- 
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able in the field. Bauman et al., like Hall and Hord, assert that this principle of program uniqueness 
removes the debate from the level of the ideal — “Should we permit reinvention?” — to the actual issue: 
“How and what are going to change while still preserving core components faithfully?" 

Reinvention of some operational components of programs is inevitable. In that context, fidelity to the 
innovative program model’s theory base is what is most critical. It is appropriate to modify, or even 
replace, procedural aspects of programs, but the theory-based intermediate outcomes must be main- 
tained. Miller and Eckholdt (1998) provide a recent example from HTV prevention. In this replication 
study, although significant operational reinvention of the program model was reported, the basic 
approaches used to impact theory- based behavioral mediators and unprotected sexual behaviors were 
maintained. The resulting impacts were similar to the effects obtained during the original program and 
its evaluation. 

Larger concerns also weigh in the direction of promoting a certain amount of adaptation, despite con- 
cerns about prograrn fidelity. Adaptation is essential in order for the community to have a role in change 
(Arthur & Blitz, 2000) and to meet needs for community ownership and involvement (Backer, 2000). 
Also, many unusual circumstances of implementation can change the nature of the implementation 
environment (Bauman et al., 1991). These often are difficult to measure, because there is a Hawthorne 
effect of measuring fidelity/adaptation. In fact, it is possible that several recent studies showing that 
high fidelity led to high outcomes are actually reflecting high attention to the fidelity process. High 
attention to the adaptation process could show the same results. 

In an early suggestion for a compromise of “modified” view. Hall et al. (1975) suggest that some change 
is acceptable up to the zone of “drastic mutation beyond which continued dilution compromises pro- 
gram integrity and effectiveness.” Several other writers have noted such possibilities as a middle ground 
between total local control and total central control of the innovation process. Overall, the position that 
seems best-justified by the weight of evidence reviewed here is one that comes from a nearly 20-year- 
old paper: 

The ultimate goal is to maintain the basic integrity of a program model while match- 
ing the innovation to the unique features of the setting and the preferences/reactions 
of the relevant setting. (Jason, Durlak, & Holton-Walker, 1984) 

Stolz (1984) calls this “implementing loosely” and acknowledges that this could result in dilution of a 
program, but it is also essential for local effectiveness. This concept is similar to one advanced by Peters 
and Waterman (1982) as among the eight key characteristics of successful American corporations: 
“simultaneous loose-tight properties.” 
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Guidelines for Balancing Program 
Fidel ity/Adaptation 

A literature review is not intended to provide detailed practice guidelines. However, program develop- 
ers and implementers are asking for help in dealing with the ultimate practical question: “how to actu- 
ally make a balance between fidelity and adaptation for a particular program in a particular setting.” A 
set of six guidelines for balancing fidelity and adaptation are offered below as a preliminary response. 

Little empirical research has been done to date on many of the issues about fidelity/adaptation balance, 
despite all the studies cited in this literature review. We do not yet know if these are precisely the steps 
for “setting the boundaries.” However, the weight of evidence suggests that these lead in the right direc- 
tion, so they become a “what-to-do-until-the-doctor-comes” set of practices. Each of the steps is based 
upon literature reviewed for this paper. 

As one example: Bauman et al. (1991) suggest that fidelity concerns should be addressed at the level 
of a causal model for how change takes place as a result of the innovation (i.e., why the program works). 
Adaptation, on the other hand, is likely to be most appropriate at the level of the specific implementa- 
tion of the program. In addition to the basic content of the program, such aspects as its level of inten- 
sity, ambiguity, and complexity, and context elements (such as how it is sponsored) also need to be 
taken into account. The three steps for fidelity/adaptation balancing presented in their study have been 
integrated into a larger six-step framework presented here. 

1. Identify and understand the theory base behind the program. 

Published literature on the program should provide a description of its theoretical underpinnings; if not, 
an inquiry to the program developer may yield this information. 

This may or may not include a logic model that describes in linear fashion how the program works. The 
theory and logic model are not in themselves core components of a program, but they can help identi- 
fy what the core components are and how to measure them. This step also identifies core values or 
assumptions about the program that can be used to help persuade community stakeholders of the pro- 
gram’s fit and importance for their environment. 

2. Obtain or conduct a core components analysis of the program. 

This will provide implementers with a roster of the main “program ingredients,” and at least some sense 
of which components are essential to likely success and which are more amenable to modification, 
given local conditions. In essence, core components analysis represents a bridge between developer and 
implementer, and between fidelity and adaptation. Ideally, the program developer or a third party will 



2002 Conference Edition 



45 



DHHS/SAMHSA/CSAP 



O 

ERIC 



48 



Finding the Balance 



already have conducted the core components analysis. If not, with good information about the program, 
an implementer can at least approximate this informally. 

As previously noted, CSAP, through its National Center for the Advancement of Prevention, is under- 
taking a large-scale core components analysis of effective and model programs. Checking to see if a 
selected program is in the database of CSAP’s National Registry of Effective Prevention Programs 
(NREPP) is a first step in determining the status of a core components analysis. For online access to 
this database, go to www.samhsa.gov/csap/modelprograms/. 

3. Assess fidelity/adaptation concerns for the particular implementation site. 

This step means determining what adaptations may be necessary, given the target population, commu- 
nity environment, political and funding circumstances, etc. And it means determining what core com- 
ponents are especially critical to address fidelity, given these same circumstances. 

4. Consult as needed with the program developer to review the above steps and how they have 
shaped a plan for implementing the program in a particular setting. 

This may also include actual technical assistance from the developer or referral to peers who have 
implemented the program in somewhat similar settings. 

5. Consult with the organization and/or community in which the implementation will take place. 

This is a process to allow fears and resistance to surface, build support for the program, and obtain input 
on how to do the implementation successfully. 

6. Develop an overall implementation plan based on these inputs. 

Include a strategy for achieving and measuring fidelity/adaptation balance for the program to be imple- 
mented, both at the initial implementation and over time. By addressing all of the complex stages of 
implementation, such a plan can increase the opportunities for making choices that shape a program, 
while maintaining good fidelity 
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Issues for Further Exploration 



Many issues remain for each of the primary audiences for fidelity and adaptation balance studies, as 
noted below: 



Issues for Researchers and Program Developers 



1. How to gather evidence on the stages of implementation, including Hdelity/adaptation balance, 
and how to make this a routine part of both research and implementation practice. 

The first step is simply to gather evidence much more routinely about the implementation process itself. 
This is not done as much as it could, or should, be (Zins, Elias, Greenberg, & Pruett, 2000). Such 
research could help to establish more precisely the appropriate balance points between fidelity and 
adaptation for different programs, target populations, or implementation settings. Even though adapta- 
tion may be inevitable, and efforts to promote fidelity difficult, only experimental research can help sort 
out how this balancing act should be treated in practice. This has not really been done to date 
(Domitrovich & Greenberg, 2000). 

Gathering more extensive evidence about fidelity/adaptation balance would help to provide a more rig- 
orous definition of what constitutes “fidelity.” There is also a need to look at the frequency of different 
types of adaptations: deletions, additions, modifications in content, changes in intensity. 

Many writers on this subject have defined questions that could be assembled together to direct future research 
inquiry. For example, Mayer and Davidson (2000) point out the following issues in the fidelity/adaptation 
debate that still need to be solved, all of which fall under this initial evidence-gathering need: 

• Is there a middle groundl 

• Should contextual factors affect the decision to favor one over the other? 

• What kinds of adaptation are benign or in fact beneficiall 

• Are fidelity and dose strength the same or different constructs? 

It may be important, too, to look at the specific role of program developers and their support organiza- 
tions in the overall process of program implementation (M. A. Pentz, personal communication, April 
2001). Program developers may simply not have the resources, or perhaps even the motivation, to sys- 
tematically disseminate information that can be used to promote effective implementation, much less to 
offer consultation to implementers. 
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For instance, developers in academic settings are rewarded for research publications, not for providing 
technical assistance, which their research grants usually do not fund. Developers who do provide tech- 
nical assistance usually have government or foundation funding to do so. Others have created private 
organizations that market their program materials and provide a revenue source that can support con- 
sultation activities. Most large-scale purchases of program materials are made by third parties (states, 
school districts, foundations, or corporations), which may or may not support technical assistance for 
implementation. Seldom are sources of funding for creating and initially validating a prevention pro- 
gram also sources for funding technical assistance to program implementers. 

On the other hand, the quality of contracted technical assistance funded as part of a local, regional, or 
national implementation effort seems variable. Often contractors are not as knowledgeable about the 
program as the original developers, but the developers are not readily available to support these efforts. 

Yet another complication comes from inherent differences between the “research standard’* and the 
“practitioner standard” for fidelity (M. A. Pentz, personal communication, April 2000). Pentz’s 
Midwest Prevention Project and McDonald’s Families and Schools Together (FAST) (L. McDonald, 
personal communication, December 2000) are among the science-based prevention programs that have 
attempted to incorporate elements of implementer-defined fidelity into the processes by which they 
encourage implementation. 

2. How to determine the sources of variance in fidelity. 

Many writers in this field speculate that this variance comes from the appropriate and vital expression 
of local needs, including the need for local control of programming. However, the research reviewed 
for this paper suggests that there are other important factors: 

• skill - some people are more skilled than others in undertaking program implementa- 
tion activities 

• effort - persistence and hard work also may account for some of the variance 

• task difficulty - some implementation tasks are simply too difficult, no matter what 
the nature of the innovation or the strategy used to implement it 

• luck - sometimes, just by happenstance, it works or it does not! 

Only further research can help to determine the relative importance of these factors more precisely. 
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3. How to understand better the differential characteristics of program implementers. 

Who adapts, what do they adapt and why, why do they preserve fidelity? For instance, the Emshoff et 
al. (under review) study shows that attention to fidelity came about because grantees were under pres- 
sure from the funder to maintain fidelity, even down to a funding mechanism that compelled adherence. 
The CSAP studies being conducted by NCAP also will address these concerns. 

4. How to address the largely unexplored question of time horizon for Hdelity/adaptation balance. 

For some innovative programs, there may be high fidelity at the beginning of the implementation peri- 
od, followed by successive stages of adaptation. These adaptations occur because environmental con- 
ditions change, or because there are changes in the scientific database about the program itself. This has 
been little explored in the research to date, which tends to look at fidelity/adaptation issues by “snap- 
shot” rather than “motion picture.” 

5. How to make the instrumentation developed for research more immediately useful for program 
implementers. 

Core components analysis, fidelity instruments, and other types of data-gathering devices described in 
this review have considerable potential for influencing field practice. This will more likely happen if 
they are presented in user-friendly formats, with adequate tie-ins to program manuals, training and tech- 
nical assistance programs, and other delivery systems. 

6. How to get program implementers more vigorously involved with all stages of program devel- 
opment. 

Implementers should provide input on how best to format and distribute instruments intended to bene- 
fit them. For instance, L. McDonald (personal communication, December 2000) reports that the 
Families and Schools Together (FAST) implementation manual includes a range of reported experi- 
ences from program implementers about “what can go wrong,” and how these challenges of previous 
implementers were addressed. 

7. How to increase the support infrastructure for implementation and Hdelity/adaptation 
research. 

One way is to provide rewards and to allot space for more data and interpretations on these subjects in 
professional journals and at conferences. Another is to encourage the development of more research 
funding programs. 
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Issues for Program Implementers 

The most important recommendations for program implementers were presented above, in terms of 
guidelines for how to address the implementation process in a practical way, including fidelity/adapta- 
tion balance. Unresolved issues tend to revolve around the mechanical aspects of fulfilling this set of 
recommendations: The following items need further attention: 

r 

1. How to deal with implementation costs. 

This includes the practical realities of limited resources in determining needed steps for implementa- 
tion and for fidelity/ adaptation balancing, even to the extent of deciding not to implement certain pro- 
grams if the financial supports are not there. 

2. How to learn about the complex knowledge base on this subject and how it can be practically 
used. 

CSAP will have a key role in responding to this issue, but individual implementers and their profes- 
sional or trade associations can also have a part in providing the local resources needed, 

3. How to get access to capacity building resources that go beyond mere knowledge, to providing 
actual technical assistance in addressing fidelity/adaptation balance and other issues posed here. 

4. How to generalize fidelity/adaptation and program implementation experiences, so that what 
is learned from an implementer’s experience with one program can be transferred to others for 
implementation in the future. 

5. How to promote leadership supportive of effective program implementation, including fideli- 
ty/adaptation balance. 

G. E. Hall (personal communication, March 2001), for instance, reports that individual teacher imple- 
menters of innovations were much more likely to implement a program successfully if their school 
principal was appropriately supportive of their efforts. 



Issues for Funders and Policymakers 

1. How to create needed learning products. 

Both developers and implementers need additional learning products in order to increase competence 
in fidelity /adaptation balance. Learning products and related items should be created and shared wide- 
ly with individual practitioners, program administrators, policymakers, and researchers as they wrestle 
with the complex issues presented here. 
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Products need to cover both basic principles and practical checklists for how to conduct a fidelity/adap- 
tation review, using the “program life cycle model” described here. Training programs need to be 
designed to provide hands-on, peer-connected training for using the new information. Finally, these 
learning resources need to be implemented in the field through larger “culture change” efforts to 
encourage the field as a whole to behave differently. These efforts will require partnerships with nation- 
al and regional substance abuse prevention and other public health organizations. 

2. How to provide needed training and technical assistance. 

A training program should be created to support prevention practitioners, program administrators, and 
community leaders in addressing the issues of fidelity/adaptation. 

Technical assistance also needs to be available on a customized basis, preferably involving program 
developers and experienced implementers. Program developers are one of the best sources for techni- 
cal assistance because of their familiarity with their programs. Peers with prior implementation experi- 
ence are also highly valuable resources. Existing systems of Federal agencies, such as the CAPT pro- 
gram in eSAP, can help to offer this technical assistance. Encouragement, and possibly funding sup- 
port, can also be offered to other groups that interface with the community of program implementers to 
provide technical assistance. 

In addition, there should be a program for “culture change” through such strategies as: 

a. Leadership statements by key organizations at their conferences, in publications, and 
on Web sites, identifying fidelity/adaptation as an issue the field needs to deal with 
more fully; 

b. Assembling “think tanks” to look at motivational, institutional, and political factors 
that might be shaped to support this culture change; and 

c. Setting up a larger range of partnerships with other Federal agencies, foundations, 
and the private sector. For instance, as mentioned, the National Institute on Drug 
Abuse has initiated a research grant program that is concerned with identifying the 
components of effective drug abuse prevention programs. The grantees in this pro- 
gram can be contacted and requested to join in the “culture change” effort described 
here, along with NIDA’s relevant project officers and senior prevention officials. 

3. How to shape funding priorities. 

To make a lasting difference in improving the quality of program implementation in substance abuse 
prevention, including the balancing of program fidelity and adaptation, funding priorities will need to 
change. Funding will need to be earmarked for these activities, both in the grants for implementation 
programs (as has occurred with the CSAP Family Strengthening initiative and the Department of 
Education Prevention Principles Program) and by creating new sources of funding for technical assis- 
tance support and field development work in this important area. 
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4. How to promote overall capacity building for the prevention Held. 

For both communities and prevention agencies, the resources recommended above can only be used 
fully if they are set in the larger context of efforts to strengthen the overall organization. Long-term 
planning for sustainability, dealing with initial needs/assets and readiness assessments, and sensitive 
balancing of program fidelity and adaptation are typical of the complex interventions that cannot be 
undertaken by weak, under-resourced entities. 

As Backer and Bare (2000) make clear, capacity building for strengthening nonprofit organizations is 
increasingly recognized by funders as an important part of improving their overall performance. The 
kinds of strategies suggested here need to be considered a component of that effort, and information 
about the types of interventions suggested here need to be offered to the capacity building providers that 
serve local entities. Such efforts can be woven into the larger systems change efforts that currently exist 
in prevention, such as the CSAP State Incentive Grants (SIG) system. 
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Next Steps 

The Center for Substance Abuse Prevention is already addressing many of the issues noted above. For 
instance, through its National Center for the Advancement of Prevention four small-scale efforts to 
explore unresolved issues in fidelity/adaptation balance are in progress: 

1. The perspectives of program developers on fidelity/adaptation are being obtained 
through discussions with several prominent developers about these issues for their 
own programs — for example, how these developers deal with enhancements and 
modifications of their programs over time. 

2. Perspectives of implementers for these same programs are being obtained through 
discussions with selected field implementers, to learn more about exactly how they 
deal with fidelity or adaptation challenges. 

3. A review of fidelity/adaptation instruments is being conducted, to lead to information 
on 

a. format and content for fidelity instruments, and how these could be better 
standardized in the substance abuse prevention field; 

b. what steps could be taken in the field to increase the use of fidelity instru- 
ments as measures of implementation quality; and 

c. how prevention practitioners/agencies and communities can use these 
instruments in practical ways. 

4. A thought paper is being prepared on how future progress in dealing with 
fidelity/adaptation balance may require obtaining information on these matters 
prospectively. 

In the end, these and other activities proposed to increase attention to the important issue of fidelity and 
adaptation balance will assist program developers and implementers to achieve desired outcomes. To 
support the field in these efforts, CSAP is coordinating a number of developmental efforts: expanding 
its core components analysis work and National Registry of Effective Prevention Programs (described 
earlier in this paper), supporting development of additional fidelity instruments, sponsoring conference 
presentations about fidelity/adaptation balance, and addressing this complex issue in other CSAP prod- 
ucts. Through these activities CSAP and the prevention field can work together to enhance the effec- 
tiveness of prevention programs, maintain accountability, and re-shape the field. 
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Progress Through Explorations in 
Other Fields 



Because fidelity/adaptation balance is such a complex issue, both CSAP and prevention researchers also 
may find it helpful to explore other content areas that could provide new insights about how to handle 
fidelity /adaptation issues. Medication compliance is one area that might be especially fruitful, because 
that field deals with the problem of people not following the medication regimen specified for them. As 
medical practitioners and public health officials know only too well, a major problem in medicine today 
is that many people fail to follow the medication regimen specified for them, either by not taking pre- 
scribed medication at all, or stopping it too soon, or taking it under conditions different from what med- 
ical science has determined will lead to the best result with the fewest side effects. 

In public health studies on this problem, it has been found that noncompliance rates of 50 percent or 
higher are common for major medical problems such as arthritis, seizure disorders, and diabetes. Half 
of patients with hypertension drop out of their medication regimen within one year, and only two-thirds 
of those who remain take adequate medication (Fenton, Blyler, & Heinssen, 1997). Fifty percent or 
higher noncompliance rates are also reported for antipsychotic medications used to treat people with 
schizophrenia. Some of the issues of family involvement and community stigma with mental illnesses 
are similar to those occurring in the substance abuse arena. 

Compliance is a collaborative relationship between the patient and practitioner where both assume 
responsibility for producing a treatment regimen to which the patient can adhere (Corrigan, Liberman, 
& Engel, 1990). Barriers to this partnership may include the patient’s individual psychopathology and 
other characteristics, characteristics of the family, the patient-clinician relationship, and treatment tech- 
niques. Eckman et al. (1992) report the successful use of modularized skill development training to deal 
with these factors, by which illness self-management including compliance can be increased. 

According to the considerable research on this subject, noncompliance is “multidetermined.” An indi- 
vidualized approach is best, which occurs in the context of an ongoing physician-patient relationship. 
This finding has many implications for the creation and maintenance of strategies for balancing fideli- 
ty and adaptation concerns in substance abuse prevention. 

The research on this subject also shows the value of interventions that might be useful in the fideli- 
ty/adaptation balance for substance abuse prevention — for instance, the use of negotiation approaches 
between patient and professional (with input from the family). Empirical and case studies, such as those 
by Sotiropoulos, Poetter, and Napholz (1999); Wirshing, Marder, Eckman, Liberman, and Mintz 
(1992); and Fenton, et al., (1997) may be examined in this regard. 
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The adult education field also may provide useful input about how to design implementation strategies 
and achieve appropriate balance of program fidelity/adaptation. In the end, implementation almost 
always involves changing the behavior of adults — teachers, prevention personnel, etc. — through an 
educational process so that they will then implement a program properly. We know, for instance, from 
studies of adult education and the writings of great theorists such as Paulo Friere, that experiential learn- 
ing and small group activities are among the strategies that work best for educating adults. 

Implementation strategies can be designed to follow these principles, as McDonald does with her FAST 
program, for instance. A set of guidelines for shaping the overall implementation process, drawing on 
the literature about implementation reviewed here, and then using such wisdom as that from the adult 
education field, might be helpful for shaping future practice. 
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Conclusion 



Bauman et al. (1991) suggest an architectural metaphor for program implementation: 

Implementing a program is like constructing a building. An architect draws upon gen- 
eral engineering principles (theory) to design a building that will serve the purposes 
for which it is designed. However, the specific building that results is strongly influ- 
enced by parameters of the building site, such as the lot size, the nature of the site’s 
geological features, the composition of the soil, the incline of the surface, the stabil- 
ity and extremes of climate, zoning regulations, and cost of labor and materials. The 
architect must combine architectural principles with site parameters to design a spe- 
cific building for a specific purpose on a specific site.... This dynamic is mirrored in 
the rough-and-tumble world of the human services. Despite excellent plans and expe- 
rience, ongoing redesign and adjustment may be necessary, (p. 34) 

The architectural plans represent a degree of purpose in building that is similar to substance abuse pre- 
vention programming and the issues discussed in this paper. For example, adaptation is most likely to 
succeed when it is highly intentional, and rarely when it is accidental or careless— “It seemed like a 
good idea at the time.” 

The foregoing review of literature and the conclusions drawn from it address the desire of those work- 
ing “in the trenches” of prevention to have usable strategies and principles for balancing program fideli- 
ty and adaptation and for dealing with the other stages of the program implementation process. There 
is a growing knowledge base on how to do this well. The recommendations made here can help both to 
increase the knowledge and to shape it into learning products and interventions that will increase the 
future quality of implementation practice in substance abuse prevention. 



2002 Conference Edition 



57 



DHHS/SAMHSA/CSAP 



Program Fidelity and Adaptation in Substance Abuse Prevention 



Appendix 




Program Fidelity and Adaptation in Substance Abuse Prevention 



CSAP's Typology of Science-Based Programi 




2002 Conference Edition 



DHHS/SAMHSA/CSAP 




Program Fidelity and Adaptation in Substance Abuse Prevention 



References 



Akerlund, K. M. (2000). Prevention program sus- 
tainability: The state’s perspective. Journal of 
Community Psychology, 28(3), 353-362. 

Allen, J. P., Philliber, S., & Hoggson, N. (1990). 
School-based prevention of teen pregnancy and 
school dropout: Process evaluation of the national 
replication of the Teen Outreach Program. American 
Journal of Community Psychology, 18(A), 505-524. 

Altman, D. G. (1995). Sustaining interventions in 
community systems: On the relationship between 
researchers and communities. Health Psychology, 
14(6), 526-536. 

Arthur, M. S., & Blitz, C. (2000). Bridging the gap 
between research and practice in drug abuse preven- 
tion through needs assessment and strategic commu- 
nity planning. Journal of Community Psychology, 
25(3), 241-256. 

Backer, T. E. (2000). The failure of success: 
Challenges of disseminating effective substance abuse 
prevention programs. Journal of Community 
Psychology, 28(3), 363-373. 

Backer, T. E. (1999). Marketing services organiza- 
tions for nonprofit arts groups: Core components and 
replication. In John S. & James L. Knight Foundation, 
Marketing the Arts. Miami: Knight Foundation. 

Backer, T. E. (1995a). Assessing and enhancing 
readiness for change: Implications for technology 
transfer. In T. E. Backer, S. L. David, & G. Soucy, 
(Eds.) Reviewing the behavioral science knowledge 
base on technology transfer. Rockville, MD: National 
Institute on Drug Abuse. 

Backer, T. E. (1995b). Dissemination and utiliza- 
tion strategies for foundations: Adding value to grant- 
making. Kansas City, MO: Ewing M. Kauffman 
Foundation. 



Backer, T. E. (1991). Drug abuse technology trans- 
fer. Rockville, MD: National Institute on Drug Abuse. 

Backer, T. E., & Bare, J. (2000). Going to the next 
level. Foundation News and Commentary, 
September/October, 38-40. 

Backer, T. E., Brown, B. S., & Howard, E. A. 
(1994). Evaluating drug abuse technology transfer: 

An analytic review. Los Angeles: Human Interaction 
Research Institute. 

Backer, T. E., David, S. L., & Soucy, G. (1995). 
Reviewing the behavioral science knowledge base on 
technology transfer. Rockville, MD: National Institute 
on Drug Abuse. 

Backer, T. E., & Diaz, E. (1999). Substance abuse 
prevention and Latino communities: A briefing paper 
for the Center for Substance Abuse Prevention. 
Northridge, CA: Human Interaction Research 
Institute. 

Backer, T. E., & Glaser, E. M. (1979). Case studies 
of nonimplementers of the Fairweather Hospital- 
Community Treatment Program. Los Angeles: Human 
Interaction Research Institute. 

Backer, T. E., & Rogers, E. M. (1999). Briefing 
paper for Dissemination Best Practices Workshop. 
Encino, CA: Human Interaction Research Institute. 

Basen-Engquist, K., O’Hara- Tompkins, N., Lovato, 
C., Lewis, M. J., Parcel, G. S., & Gingiss, P. (1994). 
The effect of two types of teacher training on imple- 
mentation of Smart Choices: A tobacco prevention 
curriculum. Journal of School Health, 64, 334-339. 

Basch, C. E. (1984). Research on disseminating and 
implementing health education programs in schools. 
Journal of School Health, 54, 57-66. 



2002 Conference Edition 



V 



DHHS/SAMHSA/CSAP 



Finding the Balance 



Battistich, V., Schaps, E., Watson, M., Solomon, 

D., & Lewis, C. (2000). Effects of the Child 
Development Project on students’ drug use and other 
problem behaviors. Journal of Primary Prevention, 
27(1), 75-99. 

Bauman, L. J., Stein, R. E., & Ireys, H. T. (1991). 
Reinventing fidelity: The transfer of social technology 
among settings. American Journal of Community 
Psychology, 79(4), 619-639. 

Berman, P. (1980). Thinking about programmed 
and adaptive implementation: Matching strategies to 
situations. In H. M. Ingram, & D. E. Mann (Eds.), 
Why policies succeed or fail. Beverly Hills, CA: Sage 
Publications. 

Berman, P. (1981). Educational change: An imple- 
mentation paradigm. In R. Lehming & M. Kane 
(Eds.), Improving schools: Using what we know. 
Beverly Hills, CA: Sage Publications. 

Berman, P, & McLaughlin, M. W. (1978). Federal 
programs supporting educational change: Vol. 8, 
Implementation and sustaining innovations (Contract 
No. R- 1589/8). Washington. DC: U.S. Office of 
Education. 

Berman, P, & McLaughlin, M. W. (1977). Federal 
programs supporting educational change: Vol. 7, 
Factors affecting implementation and continuation 
(Contract No. R- 1589/7). Washington, DC: U.S. 

Office of Education. 

Blakely, C. H., Mayer, J. P, Gottschalk, R. G., 
Schmitt, N., Davidson, W S., Rollman, D. B., & 
Emskoff, J. G. (1987). The fidelity-adaptation debate: 
Implications for the implementation of public sector 
social programs. American Journal of Community 
Psychology, 75(3), 253-269. 

Boruch, R. F., & Gomez, H. (1977). Sensitivity, 
bias and theory in impact evaluations. Professional 
Psychology, 8, 411-434. 



Botvin, G. J., Baker, E., Dusenbury, L., Botvin, E. 
M. & Diaz, T. (1995). Long-term follow-up results of 
a randomized drug abuse prevention trial in a white 
middle-class population. Journal of the American 
Medical Association, 273, 1106-1112. 

Botvin, G. J., Batson, H. W., Witts-Vitale, S., Bess, 
V., Baker, E., & Dusenbury, L. (1989). A psychosocial 
approach to smoking prevention for urban black 
youth. Public Health Reports, 704(6), 573-582. 

Brekke, J. S., & Wolkon, G. H. (1988). Monitoring 
program implementation in community mental health 
settings. Evaluation and the Health Professions, 

77(4), 425-440. 

Brown, B. B. (1998). Mziking a difference: Is jour- 
nal publication enough? Contemporary Drug Issues, 
13, 117-132. 

Calsyn, R., Tomatzky, L. G., & Dittmar, S. (1977). 
Incomplete adoption of an innovation: The case of 
goal attainment scaling. Evaluation, 4, 128-130. 

Chen, H. (1998). Theory-driven evaluations. 
Advances in Educational Productivity, 7, 15-34. 

Connell, D. B., Turner, R., & Mason, E. (1985). 
Summary of findings of the School Health Education 
Evaluation: Health promotion effectiveness, imple- 
mentation and costs. Journal of School Health, 55(8), 
316-321. 

Cook, T. D., Leviton, L. C., & Shadish, W. R. 
(1985). Program evaluation. In G. Lindzey & E. 
Aronson (Eds.), The handbook of social psychology. 
New York: Random House. 

Corrigan, P. W., Liberman, R. P, & Engel, J. 

(1990). Strategies that facilitate collaboration between 
practitioner and patient. Hospital and Community 
Psychiatry, 41, 1203-1211. 

Dalton, J. H., Elias, M. J., & Wandersman, A. 
(2000). Community psychology: Linking individuals 
and communities. Stamford, CT: Wadsworth. 



DHHS/SAMHSA/CSAP 



VI 



2002 Conference Edition 



Program Fidelity and Adaptation in Substance Abuse Prevention 



Dane, A. V., & Schneider, B. H. (1998). Program 
integrity in primary and early secondary prevention: 
Are implementation effects out of control? Clinical 
Psychology Review, J8, 23-45. 

Datta, L. E. (1981). Damn the experts and full 
speed ahead: An examination of the study of federal 
programs supporting educational change, as evidence 
against directed development for local problem solv- 
ing. Evaluation Review, 5(1), 5-32. 

Dencker, S. T., & Liberman, R. P. (1995). From 
compliance to collaboration in the treatment of schiz- 
ophrenia. International Journal of Clinical 
Psychopharmacology, 9, 75-79. 

Detrich, R. (1999). Increasing treatment fidelity by 
matching interventions to contextual variables within 
the educational setting. School Psychology Review, 
2S(4), 608-620. 

Domitrovich, C. E., & Greenberg, M. T. (2000). 
The study of implementation: Current findings from 
effective programs that prevent mental disorder in 
school-aged children. Journal of Educational and 
Psycho-logical Consultation, 77(2), 193-221. 

Durlak, J. A. (1995). School-based prevention pro- 
grams for children and adolescents. Thousand Oaks, 
CA: Sage. 

Durlak, J. A. (1997). Successful prevention pro- 
grams for children and adolescents. New York: 
Plenum Press. 

Durlak, J. A. (1998). Why program implementation 
is important. Journal of Prevention & Intervention in 
the Community, 77(2), 5-18. 

Durlak, J. A., & Wells, A. M. (1998). Evaluation of 
idicated preventive intervention (secondary preven- 
tion) mental health programs for children and adoles- 
cents. American Journal of Community Psychology, 
26, 775-802. 



Dusenbury, L., & Falco, M. (1995). Eleven compo- 
nents of effective drug abuse prevention curricula. 
Journal of School Health, 65(10), 420-425. 

Eckman, T. A., Wirshing, W. C., Marder, S. R., 
Liberman, R. P, Johnston-Cronk, K., Zimmerman, K., 
& Mintz, J. (1992). Technique for training schizo- 
phrenic patients in illness self-management: A con- 
trolled trial. American Journal of Psychiatry, 149, 
1549-1555. 

Edwards, R. W., Jumper-Thurman, P, Plested, B. 

A., Getting, E. R., & Swanson, L. (2000). Community 
readiness: Research to practice. Journal of 
Community Psychology, 28(3), 291-308. 

Elliott, D. (1998). Blueprints for violence preven- 
tion. Goldon, CO: Venture Publishing. 

Emshoff, J., Blakely, C., Gray, D., Jakes, S., 
Brounstein, P, & Coulter, J. (under review). An ESID 
case study at the Federal level. American Journal of 
Community Psychology. 

Eveland, J. D., Rogers, E. M., & Klepper, C. A. 
(1977). The innovation process in public organiza- 
tions: Some elements of a preliminary model [Final 
Report]. Ann Arbor, MI: University of Michigan, 
Department of Journalism. 

Everhart, K., & Wandersman, A. (2000). Applying 
comprehensive quality programming and empower- 
ment evaluation to reduce implementation barriers. 
Journal of Educational and Psychological 
Consultation, 77(2), 177-191. 

Felner, R. D., Phillips, R. S. C., DuBois, D., & Lease, 
A. M. (1991). Ecological interventions and the process 
of change for prevention: Wedding theory and research 
to implementation in real world settings. American 
Journal of Community Psychology, 79(3), 379-387. 

Fenton, W. S., Blyler, C. R., & Heinssen, R. K. 
(1997). Determinants of medication compliance in 
schizophrenia: Empirical and clinical findings. 
Schizophrenia Bulletin, 25(4), 637-651. 



2002 Conference Edition 



VII 



DHHS/SAMHSA/CSAP 



Finding the Balance 



Fetterman, D. M., Wandersman, A., & Kaftarian, 
SJ. (Eds.) (1996). Empowerment evaluation: 
Knowledge and tools for self-assessment and account- 
ability. Thousand Oaks, CA: Sage. 

Flay, B. R., Hansen, W. B., Johnson, A. C., Collins, 
L. M., Dent, C. W., Dwyer, K. M., Grossman, L., 
Hockstein, G., Rauch, J., Sobel, J. L., Sobol, D. F, 
Sussman, S., & Ulene, A. (1987). Implementation 
effectiveness trial of a social influences smoking pre- 
vention program using schools and television. Health 
Education Research, 2(4), 385-400. 

Fullan, M., & Pomfret, A. (1977). Research on cur- 
riculum and instruction implementation. Review of 
Educational Research, 47{2), 335-397. 

Gager, P. J., & Elias, M. J. (1997). Implementing 
prevention programs in high-risk environments: 
Application of the resiliency paradigm. American 
Journal of Orthopsychiatry, 67(3), 363-373. 

George, A. A., Hall, G. E., & Uchiyama, K. (2000). 
Extent of implementation of a standards-based 
approach to teaching mathematics and student out- 
comes. Journal of Classroom Instruction, 55(1), 8-25. 

Gingiss, P. L. (1992). Enhancing program imple- 
mentation and maintenance through a multiphase 
approach to peer-based staff development. Journal of 
School Health, 62(5), 161-166. 

Glaser, E. M., & Backer, T. E. (1980). Durability of 
innovations: How Goal Attainment Scaling programs 
fare over time. Community Mental Health Journal, 
16(2), 130-143. 

Glaser, E. M., & Backer, T, E. (1977). Innovation 
redefined: Durability and local adaptation. 

Evaluation, 4, 131-135. 

Glaser, E. M., & Backer, T. E. (1972). A clinical 
approach to program evaluation. Evaluation, I, 54-59. 



Goodman, R. M. (2000). Bridging the gap in effec- 
tive program implementation: From concept to appli- 
cation. Journal of Community Psychology, 25(3), 
309-322. 

Goodman, R. M., McLeRoy, K. R., Steckler, A., & 
Hoyle, R.H. (1993). Development of level of institu- 
tionalization scales for health promotion programs. 
Health Education Quarterly, 20(2): 161-178. 

Goodman, R. M., & Steckler, A. (1989). A frame- 
work for assessing program institutionalization. 
Knowledge, 2, 57-71. 

Goodman, R. M., & Wandersman, A. (1994). 
FORECAST: A formative approach to evaluating 
community coalitions and community-based initia- 
tives. Journal of Community Psychology, (CSAP spe- 
cial issue), 6-25. 

Gray, D. O., Emshoff, J., Jakes, S., & Blakely, C. 
(under review). ESID, dissemination research and 
community psychology: A case of partial implementa- 
tion? American Journal of Community Psychology. 

Gresham, F. M., Gansle, K. A., Noell, G. H., 

Cohen, S., & Rosenbaum, S. (1993). Treatment 
integrity of school-based behavioral intervention stud- 
ies. School Psychology Review, 22, 254-272. 

Hall, G. E. (1992). The local educational change 
process and policy implementation. Journal of 
Research in Science Teaching, 29, 877-904. 

Hall, G. E., & George, A. (2000). The use of inno- 
vation configuration maps in assessing implementa- 
tion: The bridge between development and student 
outcomes. Paper presented at the annual meeting of 
the American Educational Research Association. 

Hall, G. E., & Hord, S. M. (2001). Implementing 
change: Patterns, principles, potholes. New York: 
Allyn & Bacon. 



DHHS/SAMHSA/CSAP 



VIII 



2002 Conference Edition 



Program Fidelity and Adaptation in Substance Abuse Prevention 



Hall, G. E. (1999). Using constructs and techniques 
from research to facilitate and assess implementation 
of an innovative mathematics curriculum. Journal of 
Classroom Instruction, 34{\), 1-8. 

Hall, G. E., et al (1975). Levels of use of the inno- 
vation: A framework for analyzing innovation adop- 
tion. Journal of Teacher Education, 26, 52-56. 

Hallfors, D., Sporer, A., Pankratz, M., & Godette, 
D. (2000). D rug- Free Schools Survey: Report of 
results. Chapel Hill, NC: University of North 
Carolina. 

Hansen, W. B. (1999). Fidelity: Snatching victory 
from the jaws of defeat. Unpublished paper. 

Hansen, W. B., Graham, J. W., Wolkenstein, B. H., 
& Rohrbach, L.A. (1991). Program integrity as a 
moderator of prevention program effectiveness: 
Results for fifth grade students in the Adolescent 
Alcohol Prevention Trial. Journal of Studies on 
Alcohol, 52(6), 568-579. 

Hansen, W. B., & McNeal, R. B. (1999). Drug edu- 
cation practice: Results of an observational study. 
Health Education Research, I4{\), 85-97. 

Hawkins, J. D., & Arthur, M. W. (2001). Diffusion 
of risk and protection-focused prevention in states 
and communities. Seattle: University of Washington, 
Social Development Research Group. 

Jason, L. A., Durlak, J. A., & Holton- Walker, E. 
(1984). Prevention of child problems in the schools. 

In M.C. Roberts & L. Peterson (Eds.), Prevention of 
Problems in Childhood. New York: Wiley. 

Kretzmann, J., & McKnight, J. (1993). Building 
communities from the inside out: A path toward find- 
ing and mobilizing a community's assets. Chicago: 
ACTA Publications. 

Kumpfer, K. L., Whiteside, H. O., & Wandersman, 
A. (1997). Community readiness for drug abuse pre- 
vention: Issues, tips and tools. Rockville, MD: 
National Institute on Drug Abuse. 



Kumpfer, K. L., Shur, G. H., Ross, J.G., Bunnell, 

K. K., Librett, J. J., & Millward, A. R. (1993). 
Measurements in prevention: A manual on selecting 
and using instruments to evaluate prevention pro- 
grams. Rockville, MD: Center for Substance Abuse 
Prevention 

Lamb, S., Greenlick, M. R., & McCarty, D. (Eds.) 
(1998). Bridging the gap between practice and 
research: Forging partnerships with community-based 
drug and alcohol treatment. Washington, DC: 

National Academy Press. 

Leithwood, K. A., & Montgomery, D.J. (1980). 
Evaluating program implementation. Evaluation 
Review, 4(2), 193-214. 

Mayer, J. P, Blakely, C. H., & Davidson, W, S. 
(1986). Social program innovation and dissemination: 
A study of organizational change processes. Policy 
Studies Review, 6(2), 273- 287. 

Mayer, J. P, & Davidson, W.,S. (2000). Dissemina- 
tion of innovation and social change. In J. Rappaport 
& E. Seidman (Eds.), Handbook of community psy- 
chology. New York: Kluwer Academic/Plenum 
Publishers. 

McCormick, L. K., Steckler, A., & McLeroy, K. R. 
(1995). Diffusion of innovation in schools: A study of 
adoption and implementation of school-based tobacco 
prevention curricula. American Journal of Health 
Promotion, 9, 210-219. 

McDonald, L. (1998). FAST team trainer manual 
steps to certification. Madison, WI: Edgewood 
College, FAST Training and Evaluation Center. 

McGrew, J. H. & Bond, G. R. (1995). Critical 
ingredients of Assertive Community Treatment: 
Judgment of the experts. Journal of Mental Health 
Administration, 22, 113-126. 

Miller, R. L., & Eckholdt, H. M. (1998). HIV preven- 
tion with male prostitutes and patrons of hustler bars: 
Replication of an HTV preventive intervention. Ameri- 
can Journal of Community Psychology, 26, 97-131. 



2002 Conference Edition 



IX 



DHHS/SAMHSA/CSAP 



Finding the Balance 



Getting, G. R., Donnermeyer, J. F., Plested, B. A., 
Edwards, R. W., Kelly, K., & Beauvais, F. (1995). 
Assessing community readiness for prevention. The 
International Journal of Addictions, 30(6), 659-683. 

Palumbo, D. J., Maynard-Moody, S., & Wright, P 
(1984). Measuring degrees of successful implementa- 
tion: Achieving policy versus statutory goals. 
Evaluation Review, 8, 45-74. 

Palumbo, D. J., & Oliverio, A. (1989). Implemen- 
tation theory and the theory-driven approach to validi- 
ty. Evaluation and Program Planning, 12, 337-344. 

Parcel, G. S., Ross, J. G., Lavin, A. T., Portnoy, B., 
Nelson, G. D., & Winters, F. (1991). Enhancing 
implementation of the teenage health teaching mod- 
ules. Journal of School Health, 67(1), 35-38. 

Pentz, M. A. (2000). Institutionalizing community- 
based prevention through policy change. Journal of 
Community Psychology, 25(3), 257-270. 

Pentz, M. A., Trebow, E. A., Hansen, W. B., 
MacKinnon, D. P, Dwyer, J. H., & Johnson, C. A. 
(1990). Effects of program implementation on adoles- 
cent drug use behavior. Evaluation Review, 74(3), 
264-289. 

Peters, T. & Waterman, R. H. (1982). In search of 
excellence. New York: Harper & Row. 

Roberts-Gray, C., & Gray, T. (1983). Implementing 
innovations: A model to bridge the gap between diffu- 
sion and utilization. Knowledge: Creation, Diffusion, 
Utilization, 5(2),: 213-232. 

Rogers, E. M. (1995a). Diffusion of drug abuse 
prevention programs: Spontaneous diffusion, agenda 
setting, and reinvention. In T. E. Backer, S. L. David, 
& G. Soucy (Eds.) Reviewing the behavioral science 
knowledge base on technology transfer. Rockville, 
MD: National Institute on Drug Abuse. 

Rogers, E. M. (1995b). Diffusion of innovations. 
New York: Free Press. 



Rohrbach, L. A., D’Onofrio, C. N., Backer, T. E., & 
Montgomery, S. B. (1996). Diffusion of school-based 
substance abuse prevention programs. American 
Behavioral Scientist, 39(7), 919-934. 

Rohrbach, L. A., Graham, J. W., & Hansen, W. B. 
(1993). Diffusion of a school-based substance abuse 
prevention program: Predictors of program implemen- 
tation. Preventive Medicine, 22, 237-260. 

Scheirer, M. A. (1982). Program implementation: 
The organizational context. Beverly Hills, CA: Sage. 

Scheirer, M. A. (1996). A user’s guide to program 
templates: A new tool for evaluating program content. 
New Directions for Evaluation, No. 72. 

Scheirer, M. A. & Rezmovic, E. (1983). Measuring 
the implementation of innovations. Evaluation 
Review, 7, 599-633. 

Smith, D. W., Stecker, A., McCormick, L. K., & 
McLeroy, K. R. (1995). Lessons learned about dis- 
seminating health curricula to schools. Journal of 
Health Education, 26, 37-43. 

Sobol, D. F., Rohrbach, L. A., Dent, C. W., 

Gleason, L., Brannon, B., Johnson, C. A. & Ray, B. 
R.(1989). The integrity of smoking prevention cur- 
riculum delivery. Health Education Research, 4(1), 
59-67. 

Sorenson, J. L., Hall, S. M., Loeb, P., Allen, T., 
Glaser, E. M., & Greenberg, P. D. (1988). 
Dissemination of a job seekers’ workshop to drug 
treatment programs. Behavior Therapy, 19, 143-155. 

Sotiropoulos, P., Poetter, K. & Napholz, L. (1999). 
A case study of psychiatric medication noncompli- 
ance. Journal of the University of Arizona College of 
Nursing, September, 1-7. 

Spoth, R., Redmond, C., Kahn, J., & Shin, C. 
(1997). A prospective validation study of inclination, 
belief and context predictors of family-focused pre- 
vention involvement. Family Process, 36, 403-429. 



DHHS/SAMHSA/CSAP 



X 



2002 Conference Edition 



Program Fidelity and Adaptation in Substance Abuse Prevention 



Spoth, R., Goldberg, C., & Redmond, C. (1999). 
Engaging families in longitudinal preventive interven- 
tion research: Discrete-time survival analysis of 
socioeconomic and social-emotional risk factors. 
Journal of Consulting and Clinical Psychology, 67(1), 
157-163. 

Spoth, R., Redmond, C., & Lepper, H. (1999). 
Alcohol initiation outcomes of universal family- 
focused preventive interventions: One- and two-year 
follow-ups of a controlled study. Journal of Studies 
on Alcohol, Supp. 13 [March], 103-111. 

Spoth, R., Redmond, C., & Shin, C. (1998). Direct 
and indirect latent-variable parenting outcomes of two 
universal family-focused preventive interventions: 
Extending a public health-oriented research base. 
Journal of Consulting and Clinical Psychology, 55(2), 
385-399. 

Stolz, S. B. (1984). Preventive models: 

Implications for a technology of practice. In M. C. 
Roberts & L. Peterson (Eds.), Prevention of problems 
in childhood. New York: Wiley. 

Tobler, N. S., & Stratton, H. H. (1997). 
Effectiveness of school-based drug prevention pro- 
grams: A meta-analysis of the research. Journal of 
Primary Prevention, 75(1), 71-128. 



Tomatzky, L. G., & Klein, K. J. (1982). Innovation 
characteristics and innovation adoption-implementa- 
tion: A meta-analysis of findings. IEEE Transactions 
on Engineering Management, 29, 28-45. 

Tomatzky, L. G., Fergus, E., Avellar, J., & 
Fairweather, G. W. (1980). Innovation and social 
process. Elmsford, NY: Pergamon Press. 

Tricker, R., & Davis, L. H. (1988). Implementing 
drug education in the schools: An analysis of the costs 
and teacher perceptions. Journal of School Health, 
55(5), 181-185. 

Wirshing, W. C., Marder, S. R., Eckman, T. A., 
Liberman, R. P, & Mintz, J. (1992). Acquisition and 
retention of self-management of illness skills in 
chronic schizophrenic outpatients. Psychopharmacol- 
ogy Bulletin, 28, 241-245. 

Yin, R. (1978). Changing urban bureaucracies: 
How new practices become routinized. Santa Monica, 
CA: The RAND Corporation. 

Zins, J. E., Elias, M. J., Greenberg, M. T, & Pruett, 
M. K. (2000). Promoting quality implementation in 
prevention programs. Journal of Educational and 
Psychological Consultation, 77(2), 173-174. 



r 



2002 Conference Edition 



XI 



DHHS/SAMHSA/CSAP 



Program Fidelity and Adaptation in Substance Abuse Prevention 



In Appreciation 

Thanks are extended to Dr. Steven Schinke, Columbia University, and Dr. Nancy Jacobs, John Jay 
Criminal Justice Research Center, City University of New York, for their many contributions to this 
paper, produced as one part of our work together on the Center for Substance Abuse Prevention’s 
National Center for the Advancement of Prevention. The following individuals served very helpfully as 
expert reviewers of the paper in several draft stages, providing both conceptual and practical input: 

Craig Blakely, Texas A&M University 

James Derzon, Pacific Institute for Research and Evaluation 

Joseph Durlak, University of Illinois 

Delbert Elliott, University of Colorado 

James Emshoff, Georgia State University 

Gene Hall, University of Nevada 

William Hansen, Tangle wood Research 

Lynn McDonald, University of Wisconsin 

Mary Ann Pentz, University of Southern California 

Everett Rogers, University of New Mexico 

Richard Spoth, Iowa State University 

Sue Thau, Consultant 

Staff of the Human Interaction Research Institute who supported development of this paper included 
Elizabeth Howard, Kimarie Klahn, U. Sue Koone, Cynthia Kunz, and Terri Ruddiman. 



2002 Conference Edition 



XIII 



DHHS/SAMHSA/CSAP 



Program Fidelity and Adaptation in Substance Abuse Prevention 

FAXBACK FEEDBACK 



“Program Fidelity and Adaptation in Substance Abuse Prevention, A State-of-the-Art Reveiw” 
is intended for use by prevention practitioners and professionals at the state and local levels. 

Please rate your satisfaction with following dimensions of this publication. 



CONTENT 


Very 

Dissatisfied 


Somewhat 

Dissatisfied 


Neutral 


Somewhat 

Satisfied 


Very 

Satisfied 


1 . Relevance of the information 












2. Accuracy of the information 












3.Timeliness of the information 













FORMAT 


Very 

Dissatisfied 


Somewhat 

Dissatisfied 


Neutral 


Somewhat 

Satisfied 


Very 

Satisfied 


1. Overall Presentation 












2. Readability 












3* Organization 













UTILITY 



I.Will be useful next year 



Very 

Dissatisfied 



Somewhat 

Dissatisfied 



Neutral 



Somewhat Very 
Satisfied Satisfied 



2. Useful to selected asudience 



3. Product is useful given 
expected expense 



COMMENTS: 



SUGGESTED TOPICS FOR SIMILAR PRODUCT DEVELOPMENT: 



AFFILIATION: 



POSITION: 



Please fax your feedback to the National Center for the Advancement of Prevention at (301) 984-6095. 



How to obtain this document: 



This document can be obtained online at Internet sites sponsored by the 
Federal Center for Substance Abuse Prevention (CSAP): 

CSAP Decision Support System (DSS) Web site: 
vwvw.preventiondss.org 

CSAP Model Programs Web site: 
vwvw.modelprograms.samhsa.gov/ 

CSAP Prevention Pathways Web site: 
vwvw.sam h sa.gov/p revention path ways/ 




71 




ilipi 

iiiii 




S u b sta nee Abus e a n d M enx.a\ 
Services Ad?Tiinisfr3tion 



o 

ERIC 

hfiiiiTiiimrrraaJ 




BEST COPY AVAILABLE 



r: 






Finding the Balance: 

o 

- X 

Program Fidelity and Adaptation 
in Substance Abuse Prevention 

■ ...... f - 

Executive Summary of a State-of-the-Art Review 



o 

ERIC 






% 






U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES?; 
Substance Abuse and Mental Health Services Administration^ 
Center for Substance Abuse Prevention 
www.samhsa.gov 



BEST COPY AVAILABLE 




Finding the Balance: 



Program Fidelity and 
Adaptation in Substance 
Abuse Prevention _ 

Executive/Summary 

of a Stqte-of-the-Art Riview 

/ 

/ 

/ 




Finding the Balance 



Acknowledgments 

This document marks CSAP’s progress towards fulfilling its commitment to bringing effective, science- 
based prevention to every community across the country. 

One of several in a new series developed by CSAP, this conference-edition document articulates 
CSAP’s policy direction and guidance to the field on prevention practices that we know can be effective 
in creating positive change. These documents are products of the collaboration among CSAP, States, the 
National Prevention Network (NPN), Community Anti-Drug Coalitions of America (CADCA), and rep- 
resentatives from both the research and practice communities. As such, they represent our collective 
best thought and guidance on effective prevention. 

As CSAP continues to build its National Dissemination System to identify and encourage effective pre- 
vention and provide capacity building opportunities for states and communities, these documents will 
evolve in nature and content. Throughout this evolutionary process, CSAP will collaborate with States, 
intermediary organizations, and community practitioners, and will listen and learn about the challenges 
encountered in moving the field of prevention forward. CSAP will integrate this feedback, developing 
new guidance to support the field as it continues to grow and advance. 

CSAP is proud of our collaboration with the field and the documents that have resulted. We especially 
would like to acknowledge the significant contributions of Thomas Backer, Ph.D., senior social scientist 
affiliated with CSAP's National Center for the Advancement of Prevention (Contract No. 277-99-6023). 



DHHS/SAMHSA/CSAP 



II 



2002 Conference Edition 



Program Fidelity and Adaptation in Substance Abuse Prevention 



Table of Contents 



Executive Summary 1 

What Do We Mean by Fidelity and Adaptation? 4 

Why are Program Fidelity and Adaptation Important? 8 

A Resurgence in Interest 9 

What Does Research on Fidelity and Adaptation Tell Us? 12 

Main Conclusions from the Literature Review 13 

What Does This Mean to Us? 15 

Additional Research Issues 17 

Next Steps 23 

Conclusion 25 

Appendix I- VI 

CSAP’s Typology of Science-Based Programs Ill 

References IV 

Feedback Faxback Form VII 



2002 Conference Edition 



III 



DHHS/SAMHSA/CSAP 



Program Fidelity and Adaptation in Substance Abuse Prevention 



Finding the Balance: 

Program Fidelity and Adaptation 
in Substance Abuse Prevention 



** Implementing a program is like constructing a building. An architect 
draws upon general engineering principles (theory) to design a building 
that will serve the purposes for which it is designed. However, the specif- 
ic building that results is strongly influenced by parameters of the build- 
ing site, such as the lot size, the nature of the site's geological features, 
the composition of the soil, the incline of the surface, the stability and 
extremes of climate, zoning regulations, and cost of labor and materials. 

The architect must combine architectural principles with site parameters 
to design a specific building for a specific purpose on a specific site. 

...This dynamic is mirrored in the rough-and-tumble world of the human 
services. Despite excellent plans and experience, ongoing redesign and 
adjustment may be necessary. ” 

(Bauman, Stein & Ireys, 1991, p. 34) 

For the architects and builders of substance abuse prevention programs — whether pro- 
gram developers, the practitioners and community leaders who implement the programs, 
researchers, funders, or policymakers — balancing program fidelity and adaptation is a 
great challenge. 

How do we find the right balance between maintaining the fidelity of a science-based 
program and adapting that program to reflect the circumstances of the community 
where it will be implemented? What can we learn from research on issues of fidelity/ 
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adaptation balance that can help us in real life? How can we test the boundaries: How 
much fidelity is essential? How much adaptation is possible? 

The Center for Substance Abuse Prevention (CSAP), through its National Center for the 
Advancement of Prevention (NCAP), is leading the way in finding answers to these 
questions. CSAP sponsors a variety of studies on fidelity and adaptation issues and is 
helping to apply the lessons that emerge from these studies. The better we understand 
the complexities of fidelity and adaptation balance, the more strategically we can 
approach these issues in our work — and the better the prevention field as a whole can 
deal with the challenges of balancing program fidelity and adaptation. 

As a key building block in this process, Dr. Thomas E. Backer, one of NCAP’s principal 
senior social scientists, conducted a thorough review of research studies on fidelity and 
adaptation balance. In a paper prepared for CSAP titled “Balancing Program Fidelity 
and Adaptation in Substance Abuse Prevention: A State-of-the-Art Review,” Backer sur- 
veys 117 published and unpublished studies spanning more than 25 years. He structures 
the literature review using a conceptual model shown as Program Implementation 
Stages (see Exhibit 1). In addition. Backer provides an extensive list of references to 
guide researchers to the full body of literature surveyed for his report. 

From his synthesis of the body of research. Backer draws this fundamental conclusion: 

Attention to BOTH program fidelity and adaptation during the complex process of 
program implementation is critical to successful, sustained implementation of sci- 
ence-based substance abuse prevention programs. 

In addition to presenting this and several other conclusions from the literature. Backer 
proposes an initial set of guidelines for program implementers. He also outlines unre- 
solved issues that require attention from each of the primary audiences for this work: 
program developers and researchers, implementers, funders, and policymakers. 

Backer’s (2001) main conclusions and recommendations are described in this Executive 
Summary of his comprehensive paper prepared for CSAP. 
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EXHIBIT 1: 

STAGES IN SUBSTANCE ABUSE 
PREVENTION PROGRAM IMPLEMENTATION 

( Program Development/Validation/Dissemination) 

Program Adoption 

Needs and Assets Assessment 

Readiness Assessment 

Program Assessment 
Theory and/or Logic Model 
Core Components Analysis 
Developer Consultation 

Program Implementation 
Fidelity/Adaptation Balance 
Implementation Process 
Implementation Setting 
Client Characteristics 
Larger Context 

Evaluation 
Fidelity Instrument 
Process Evaluation 
Outcome Evaluation 

Sustainability 

Revisiting Fidelity/Adaptation 
Routinization 

(Feedback to Developer/Program Modification) 
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What Do We Mean by Fidelity and Adaptation? 

Terminology varies within the body of research on fidelity and adaptation and related 
issues. Here are definitions for key terms used in the CSAP studies: 

Program Fidelity: the degree of fit between the developer-defined components of a 
substance abuse prevention program, and its actual implementation in a given orga- 
nizational or community setting. 

How well does the implementation of a promising, effective or model program 
match the specifications of the original? The program’s elements are specified in a 
program manual, curriculum, or core components analysis. In essence, the develop- 
er provides a “recipe” for replicating the program, and also describes the fidelity 
instrument(s) for measuring the “fit” of its implementation. 

What we call “fidelity” is also called program “adherence” or “integrity” in some of 
the literature on this subject. Terms from medicine, such as “dosage,” “strength of 
treatment” “intensity,” “exposure,” are sometimes used to discuss the overall degree 
of fidelity (Boruch & Gomez, 1977), (Pentz, 2001). Also drawing from the medical 
field, “compliance” is sometimes a metaphor for fidelity/adaptation balance, mean- 
ing the extent to which a particular protocol or regimen is followed. 

Program Adaptation: deliberate or accidental modification of the program, includ- 
ing (a) deletions or additions (enhancements) of program components, (b) modifica- 
tions in the nature of the components that are included, (c) changes in the manner or 
intensity of administration of program components called for in the program manu- 
al, curriculum, or core components analysis, or (d) cultural and other modifications 
required by local circumstances. 

Adaptation is sometimes called “reinvention.” (Rogers, 1995) Other notable varia- 
tions include “mutual” adaptation, which involves adaptation both of the innovation 
and of the organization or community in which it is implemented; “cosmetic” 
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adaptation, when only the name or some superficial element of a program is 
changed to promote local ownership; and “accidental” adaptation, involving 
changes that implementers do not realize they have made. 

Fidelity/Adaptation Balance: a dynamic process, often evolving over time, by 
which those involved with implementing a science-based substance abuse preven- 
tion program address both the need for fidelity to the original program and the need 
for local adaptation. 

There are typically two places in the implementation process when seeking balance 
occurs: (1) at the front end, with the decision to adopt an effective program that 
needs some modification to fit local circumstances; and (2) during implementation, 
if the expected outcomes are not being achieved. In addition, strategically revisiting 
fidelity/adaptation balance may be an important element in program sustainability. 

Core Components: Those elements of a program that analyis shows are most likely 
to account for its positive outcomes. 

Some programs comprise essentially only their core components. Others have dis- 
cretionary or optional components which can be deleted without major impact on 
the program’s effectiveness, or which are not essential for the program’s main target 
audience. 

Core components are the essential, or main, ingredients in the recipe. CSAP is cur- 
rently conducting a core components analysis of effective and model programs, 
which is expected to be completed in early 2002, as part of developing and main- 
taining the National Registry of Effective Prevention Programs (NREPP). 

This work started with analysis of fidelity instruments for the seven CSAP High 
Risk Youth model programs. From this, a method was developed for identifying 
core components and rating their degree of implementation. Data for eight programs 
were compiled and sent to independent reviewers for appraisal and subsequent 
modification. The revised method is being used to appraise all of the NREPP pro- 
grams (Schinke, 2000). 
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Implementation: the complex process by which a substance abuse prevention pro** 
gram is put into place in a community or organization, for use with a particular tar- 
get audience. 

As indicated in the Program Implementation Stages (Exhibit 1), this is a multi-stage 
process. The model shown is an ideal. However, not all prevention programs are 
implemented following a systematic assessment; not all are evaluated. And though 
the list of stages suggests a linear order, the occurrence of activities may vary from 
one implementation to another. 

Furthermore, definitions for what implementation includes vary within the literature. 
Here, the definition used addresses program fidelity and adaptation conceptually. In 
this context, program fidelity/adaptation is considered one element of a larger 
sequence of implementation stages. 

Science-Based Programs: CSAP defines science-based substance abuse prevention 
programs as those programs that are theory-driven, reasonably well-evaluated, and 
include program activities related to the theory. 

Effective programs consistently achieve positive outcomes while promising pro- 
grams show at least some positive outcomes. Finally, there are model programs, 
which are science-based and effective and are readily available for dissemination. 
Technical assistance for model programs is available from the program developers. 
(See Exhibit 2 on the next page and the Typology included in the Appendix.) 
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EXHIBIT 2: 



Science-Based Programs 

• Theory-Driven 

• Program Activities Related to Theory 

• Reasonably Well Implemented & Evaluated 




Promising 

Some Positive 
Outcomes 




Consistently Positive Outcomes 
Strongly Implemented & Evaluated 




— M 

Model 

• Availability for Dissemination 

• Technical Assistance Available 
from Program Developers 



Note: A more detailed version of the above schemata can be found in the Appendix. 
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Why Are Program Fidelity 
and Adaptation Important? 



Each of the primary audiences for fidelity and adaptation studies has a stake in the find- 
ings. Program developers and prevention researchers are legitimately concerned that 
changes in a science-based program will dilute or even dissipate its effectiveness. 
Community leaders and prevention practitioners are equally concerned that “not one 
size fits all.” The inability to modify programs may produce local resistance; or worse, 
rigid fidelity can lead to programs that are irrelevant or even inappropriate for meeting 
community needs. Policymakers and funders struggle with defining what requirements 
for fidelity or permissions for adaptation are appropriate in guidance related to funding 
or public policy. 

Equally important, each of these groups can contribute to the improved ability of the 
prevention field as a whole to deal with the challenges of balancing program fidelity and 
adaptation. For instance, program developers and prevention researchers can use scien- 
tific analysis to identify “core components” of effective and model prevention pro- 
grams — those elements that must be maintained rigorously in order for the program to 
work. Community leaders and prevention practitioners can develop program implemen- 
tation approaches that address fidelity/adaptation balance strategically. 

Policymakers and funders can build improved standards for fidelity/adaptation balance 
into grant making and public policy about prevention programming. And all these 
groups can work together in creating partnerships to change the culture of prevention, so 
that fidelity/adaptation balance issues can be addressed intentionally when implementing 
science-based programs in the field. 

In a sense, all these groups want to know “what the boundaries are.” (Emshoff, Blakely, 
Gray, Jakes, Brounstein & Coulter, under review) There is considerable scientific evi- 
dence that many science-based prevention programs still produce positive results despite 
significant adaptation. Some adaptations are, in fact, necessary for program success, 
given widely varying circumstances in different organizations or communities. 
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However, there is also significant evidence that the greatest impact from these programs 
results when there is program fidelity with respect to certain key elements. Program 
enhancements added to a rigorously implemented “core” program may yield particularly 
good outcomes. But some adaptations are undesirable, whether deliberate or accidental. 

Efforts both to promote fidelity and to engage in adaptation already happen all the time, 
of course, but often not as strategically or effectively as would be possible with better 
guidance. The conclusions and guidelines that emerge from the literature review can 
help provide a better balance between these concerns. 



A Resurgence in Interest 

Issues of fidelity/adaptation balance related to program implementation have been stud- 
ied and discussed since at least the 1960’s, when applied research in the education and 
human services fields became more widespread, and the field of program evaluation was 
bom. The impetus came from efforts to explain why so many evaluation results with 
previously validated programs were null or inconsistent — raising the possibility of an 
effect from the implementation of the program, rather than the program itself. 

In recent years, there has been renewed interest in this subject. Why? For one reason: 
there simply are more effective substance abuse prevention programs available, resulting 
in quantitatively greater need to look at fidelity/adaptation concerns. 

Research over the last 40 years also has helped us to understand better the complexities 
of implementation for educational and substance abuse programs. This leads to greater 
chances for effective intervention. Technologies for managing the implementation 
process are now available, some of them guided by sophisticated behavioral and man- 
agement science concepts. 
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Furthermore, substance abuse prevention programs are subjected to more change, more 
complexity, and more demands for producing concrete results, while at the same time, 
substance use persists for certain populations This leads to greater urgency for commu- 
nities to learn from studies in this field. 

Funders are demanding more program accountability, which is easier with promising, 
effective and/or model programs. Funding programs, such as CSAP’s State Incentive 
Grants, the Department of Education’s “Principles of Prevention” grants, and those 
through the Drug Free Schools and Communities, now call for the use of such science- 
based prevention programs. 

Also, there is increasing recognition that informally or poorly run programs can do more 
harm than good in organizational or community settings. Loss of program fidelity can 
sometimes lead to chaos, because once the program has been modified, no one quite 
knows how it will operate or what unexpected consequences it will produce. 

On the other hand, prevention agencies and communities also face pressures for adapta- 
tion. To many observers, program fidelity represents a “top-down” approach to imple- 
menting a substance abuse prevention program, while adaptation represents a “bottom- 
up” approach that may have broader political appeal. Some communities may simply 
have an aversion to “copying;” they want to be “different, ” to “be creative.” 

Even in the best-developed programs, there are often differences in community environ- 
ments and target populations that really do necessitate certain kinds of modifications. 
Many of these ultimately are about resource limits, as in the example about Life Skills 
Training ( Exhibit 3). If a teacher has only so many classroom hours to devote to a pre- 
vention program, either a longer program is modified, or it is not used at all. 
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EXHIBIT 3: 

Fidelity/Adaptation Balance 
in the “Real World” of Program Implementation 

In Life Skills Training (LST), a well-validated program developed by Gilbert 
Botvin at Cornell University (Botvin et al., 1995, 1989), one component 
involves middle school students talking in their LST sessions about tobacco 
advertisements. LST’s curriculum suggests that students cut out tobacco ads at 
home and bring them to class for discussion. Sometimes teachers will also ask 
students to draw their own humorous ads and share them during the LST ses- 
sions. This is an example of program adaptation (specifically, an enhancement, 
since it adds something to LST that does not substitute for any existing compo- 
nent). 

Some teachers drop the LST curriculum session on relaxation, because they fear 
losing control of their students during this enforced “quiet time.” Others drop 
sessions because of time constraints. LST is an eight-session program; but in 
some schools, only six sessions are available for completing the program. 

These adaptations can have a significant bearing on whether the LST experience 
of some students is really the same as that of others. For LST, as for all science- 
based substance abuse prevention programs, having a more unified set of guid- 
ance for fidelity/adaptation balance (and indeed, for the entire process of pro- 
gram implementation) would likely improve overall program impact. 

As with other substance abuse prevention programs, many LST implementers 
are trying to determine what they believe are its core components, selecting ones 
that work for them, deleting those that do not, and adding elements they think 
will improve the program. This program adaptation will go on, says the weight 
of evidence, regardless of what program developers or researchers think about it. 
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What Does Research 

on Fidelity and Adaptation Tell Us? 

The stages of program implementation depicted in Exhibit 1 provide a conceptual 
framework for organizing the review of wide-ranging literature on the question: “How 
can the balance between program fidelity and adaptation best be set in order to promote 
successful implementation of science-based substance abuse prevention programs?” 

Even though the seven stages noted above are not always followed precisely in the real 
world of prevention programming — sometimes stages are skipped, or minimized, or 
conducted simultaneously, or re-arranged — this paradigm presents a useful way of con- 
ceptualizing and evaluating whatever process does get followed. 

A review of the literature on the concept of fidelity/adaptation balance in particular, 
which is one sub-part in those stages, shows that research, practice, and theoretical dis- 
cussions about fidelity/adaptation balance are colored by the complex human dynamics 
of cultural differences. These include, for example, differences between academically 
based program developers, who are often researchers as well; and community-based 
program implementers, who are sometimes grassroots advocates as well. 

Differences may also arise from competitiveness, (e.g., in academic publications; also, 
some science-based prevention programs are now sold commercially). Differences may 
also arise from communication difficulties among people who come from these varied 
backgrounds. These differences are sometimes further complicated by the number of 
subject fields in which fidelity/adaptation issues have been discussed, each of which has 
its own nomenclature. 

Some of the distinctions are somewhat artificial. Certainly there is a vested interest in 
the “purity” of a program model by its creator, and a vested interest in local control by 
organizations and communities that are implementing the model programs. But increas- 
ingly, both “sides” recognize that the key to program success (as defined by both par- 
ties) is to strike a balance between fidelity and adaptation. The remaining problems of 
human dynamics are likely a matter of communication and coordination. 
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Main Conclusions 
from the Literature Review 

The most important conclusion drawn from the literature review is that attention to 
BOTH fidelity and adaptation is essential for successful implementation of evi- 
dence-based substance abuse prevention programs. That is, fidelity/adaptation is not 
a continuum upon which each specific implementation of a substance abuse prevention 
program falls. Rather, fidelity/adaptation balance is concerned with the complex, 
dynamic interaction between a program and its environment. Science and experience say 
that maximum success requires attention to both fidelity and adaptation. 

To date, only limited attention has been paid to fidelity, adaptation, and other compo- 
nents of program implementation in both the science and practice of substance abuse 
prevention programming. Yet science shows that dealing with implementation issues is 
critical to program success. Evaluators have even coined a term — “Type III error” — to 
label the significant number of cases where researchers conclude that a program is not 
effective, but, in fact, what they observed was the result of the program not being imple- 
mented properly. 

Even with science-based programs and good implementation strategies (including atten- 
tion to fidelity/adaptation balance), there is no guarantee that a program will lead to sig- 
nificantly improved outcomes, such as development of resistance skills, less use of 
drugs or alcohol than comparison groups, etc. Evaluating implementation efforts and 
measuring the balance set between fidelity and adaptation have meaning only in the 
context of outcomes, and these too must be measured by research. 

Many science-based prevention programs now widely used in the field have not been 
replicated. They have been shown to work in an initial setting, but they have not been 
verified as working in a wide range of environments, or as implemented by people other 
than their developers. In some cases, even when implemented well, results may not be 
verified due to lack of systematic data gathering. 
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The research evidence about effective program implementation and fidelity/adaptation 
balance is far from complete. It is filled with contradictions and uncertainties. There are 
also some basic issues of terminology as well as methodology to be resolved. Different 
investigators use different terms to discuss this topic. Further, the measures used by pro- 
gram developers to determine program fidelity vary in scope and precision. 

Despite these shortcomings and inconsistencies, however, the weight of evidence sug- 
gests that conflicts that once divided program developers concerned with fidelity and 
program implementers concerned with adaptation are now outdated. These discussions 
still appear at times both in the literature and in the field, but the view that fidelity/adap- 
tation is a dynamic concept in which both elements are needed for program success 
makes the early arguments on this subject irrelevant. 

Hall & Hord (2001) assert that asking the question “Is adaptation desirable?” is the 
wrong question. Adaptation will happen, so the questions to be asked instead are: “How 
much?” and “When is a program’s content damaged?” 

Bauman, Stein, & Ireys (1991) refer to the “principle of program uniqueness” — that in 
its actual implementation, any program will have some unique elements because of the 
unique characteristics of the environment. They point out that many programs are creat- 
ed under unusual conditions (special funding, cheirismatic leaders, etc.) that are not 
widely available in the field. This principle of program uniqueness removes the debate 
from the level of the ideal — “Should we permit reinvention?” — to the actual issue: 

“How and what is going to change while still preserving core components faithfully?” 

Reinvention of some operational components of programs is inevitable. In that context, 
fidelity by the innovative program to the prototype’s theory base is what is most critical. 
It is appropriate to modify, or even replace, procedural aspects of programs, but the the- 
ory-based intermediate outcomes must be maintained. 

Larger concerns also weigh in the direction of promoting a certain amount of adaptation. 
Adaptation is essential in order for the community to have a role in change (Arthur & Bhtz, 
2000), and to meet needs for community ownership and involvement (Backer, 2000). Also, 
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unusual circumstances of implementation can change the nature of the implementation 
environment, and these are often difficult to measure. 

Overall, the position that seems best justified by the weight of evidence reviewed in the 
literature is one that comes from a nearly 20-year-old paper: 

""The ultimate goal is to maintain the basic integrity of a program model 
while matching the innovation to the unique features of the setting and 
the preferences/reactions of the relevant population. ” 

(Jason, Durkal & Holton- Walker, 1984) 



What Does This Mean to Us? 

A literature review is not intended to provide detailed practice guidelines. However, pro- 
gram developers and implementers are asking for help in dealing with the ultimate prac- 
tical question: “how to actually make a balance between fidelity and adaptation for a 
particular program in a particular setting.” As a preliminary response, Backer offers a set 
of six guidelines for balancing fidelity and adaptation. 

He also notes several issues for researchers, program developers, implementers, funders, 
and policymakers that should receive additional attention. These represent, in effect, 
additional conclusions from the literature review. 

Little empirical research has been done to date on many of the issues about 
fidelity/adaptation balance, despite all the studies cited in the full literature review. We 
do not yet know if these are precisely the steps for “setting the boundaries.” However, 
the weight of evidence suggests that these lead in the right direction, so they become a 
“what-to-do-until-the-doctor-comes” set of practices. Each of the steps is based upon lit- 
erature reviewed for the field study. 
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Guidelines for Balancing Program Fidelity/Adaptation 

1 - Identify and understand the theory base behind the program. Published literature 
on the program should provide a description of its theoretical underpinnings; if not, 
an inquiry to the program developer may yield this information. 

This may or may not include a logic model that describes in linear fashion how the 
program works. The theory and logic model are not in themselves core components 
of a program, but they can help identify what the core components are, and how to 
measure them. This step also identifies core values or assumptions about the pro- 
gram that can be used to help persuade community stakeholders of the program’s fit 
and importance for their environment. 

2 - Locate or conduct a core components analysis of the program. This will provide 
implementers with a roster of the main “program ingredients,” and at least some 
sense of which components are essential to likely success and which are more 
amenable to modification, given local conditions. In essence, core components 
analysis represents a bridge between developer and implementer, and between 
fidelity and adaptation. Ideally, the program developer or a third party will already 
have conducted the core components analysis. If not, with good information about 
the program, an implementer can at least approximate this informally. 

CSAP, through its National Center for the Advancement of Prevention, is undertak- 
ing a large-scale core components analysis of effective and model programs. 
Checking to see if a selected program is in the database of CSAP’s National 
Registry of Effective Prevention Programs (NREPP) is a first step in determining 
the status of a core components analysis. For online access to this database, go to 
www.samhsa.gov/csap/modelprograms/. 

3 - Assess fidelity/adaptation concerns for the particular implementation site. This 
step means determining what adaptations may be necessary, given the target 
population, community environment, political and funding circumstances, etc. 
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4 - Consult as needed with the program developer to review the above steps and 
how they have shaped a plan for implementing the program in a particular setting. 
This may also include actual technical assistance from the developer or referral to 
peers who have implemented the program in somewhat similar settings. 

5 - Consult with the organization and/or community in which the implementation 
will take place. This is a process to allow fears and resistance to surface, build sup- 
port for the program, and obtain input on how to do the implementation successfully. 

6 - Develop an overall implementation plan based on these inputs. Include a strate- 
gy for achieving and measuring fidelity/adaptation balance for the program to be 
implemented, both at the initial implementation and over time. By addressing all of 
the complex stages of implementation, such a plan can increase the opportunities 
for making choices that shape a program, while maintaining good fidelity. 



Additional Research Issues 

Many research issues remain for each of the primary audiences for fidelity and adapta- 
tion balance studies, as noted below: 

Issues for Researchers and Program Developers 

7 - How to gather evidence on the stages of implementation, including 
fidelity/adaptation balance, and how to make this a routine part of both research 
and implementation practice. 

More evidence about the implementation process itself is a fundamental research 
need. Such research could help to establish more precisely the appropriate balance 
points between fidelity and adaptation for different programs, target populations, or 
implementation settings. Even though adaptation may be inevitable, and efforts to 
promote fidelity difficult, only experimental research can help sort out how this 
balancing act should be treated in practice. This has not really been done to date. 
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Gathering more extensive evidence about fidelity/adaptation balance would help to 
provide a more rigorous definition of what constitutes “fidelity.” There is also a 
need to look at the frequency of different types of adaptations: deletions, additions, 
modifications in content, changes in intensity. 

It may be important, too, to look at the specific role of program developers and 
their support organizations in the overall process of program implementation 
(Pentz, 2001). Program developers may simply not have the resources, or perhaps 
even the motivation, to systematically disseminate information that can be used to 
promote successful implementation. They may have even less incentive to offer 
consultation to implementers. 

For instance, developers in academic settings are rewarded for research publica- 
tions, not for providing technical assistance, which their research grants usually do 
not fund. Developers who do provide technical assistance usually have government 
or foundation funding to do so. Others have created private organizations that mar- 
ket their program materials and provide a revenue source that can support consulta- 
tion activities. Most large-scale purchases of program materials are made by third 
parties (states, school districts, foundations, or corporations), which may or may not 
support technical assistance for implementation. Seldom are sources of funding for 
creating and initially validating a prevention program also sources for funding tech- 
nical assistance to program implementers. 

On the other hand, the quality of contracted technical assistance funded as part of a 
local, regional, or national implementation effort seems variable. Often contractors 
are not as knowledgeable about the program as the original developers. Yet another 
complication comes from inherent differences between the “research standard” and 
the “practitioner standard” for fidelity. 

2 - How to determine the sources of variance infidelity. 

Many writers in this field say that this variance comes from the appropriate and vital 
expression of local needs, including the need for local control of programming. 
However, based on the broad literature review, there are other important factors as 
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well, such as the skill and effort of the people involved in the implementation, the 
difficulty of some implementation tasks, and even luck. Only further research can 
help to determine the relative importance of these factors more precisely. 

3 - How to understand better the differential characteristics of program 
implementers. 

Who adapts, what do they adapt and why, why do they preserve fidelity? For 
instance, the Emshoff et al. study shows that attention to fidelity came about 
because grantees were under pressure from the funder to maintain fidelity. 

4 - How to address the largely unexplored question of time horizon for fidelity/adap- 
tation balance. 

For some innovative programs, there may be high fidelity at the beginning of the 
implementation period, followed by successive stages of adaptation. These adapta- 
tions occur because environmental conditions change, or because there are changes 
in the scientific database about the program itself This has been little explored in 
the research to date, which tends to look at fidelity/adaptation issues by “snapshot” 
rather than “motion picture.” 

5 - How to make the instrumentation developed for research more immediately use- 
ful for program implementers. 

Core components analysis, fidelity instruments, and other types of data-gathering 
devices described in this review have considerable potential for field practice. This 
will more likely happen if they are presented in user-friendly formats, with adequate 
tie-ins to program manuals, training and technical assistance programs, and other 
delivery systems. 

6 - How to get program implementers more vigorously involved with all stages of 
program development. 

For instance, implementers should provide input on how best to format and distrib- 
ute instruments intended to benefit them. 
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7 - How to increase the support infrastructure for implementation and fidelity/adap- 
tation research. 

One way is to provide rewards and to allot space for more data and interpretations 
on these subjects in professional journals and at conferences. Another is to encour- 
age the development of more research funding programs. 



Issues for Program Implementers 

The most important recommendations for program implementers were presented above, 
in terms of guidelines for how to address the implementation process in a practical way, 
including fidelity/adaptation balance. Unresolved issues tend to revolve around the 
mechanical aspects of fulfilling this set of recommendations: The following items need 
further attention: 

7 - How to deal with implementation costSy including the practical realities of limit- 
ed resources in determining needed steps for implementation and for fidelity/ adap- 
tation balancingy even to the extent of deciding not to implement certain programs if 
the financial supports are not there. 

2 - How to learn about the complex knowledge base on this subject and how it can 
be practically used. 

CSAP will have a key role in responding to this issue, but individual implementers 
and their professional or trade associations can also have a part in providing the 
local resources needed. 

3 - How to get access to capacity building resources that go beyond mere knowl- 
edgey to providing actual technical assistance in addressing fidelity/adaptation bal- 
ance and other issues posed here. 

4 - How to generalize fidelity/adaptation and program implementation experienceSy 
so that what is learned from an implementer's experience with one program can be 
transferred to others for implementation in the future. 
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5 - How to promote leadership supportive of effective program implementation, 
including fidelity/adaptation balance. 

Hall (2001) reports that individual teacher implementers of innovations were much 
more likely to implement a program successfully if their school principal was 
appropriately supportive of their efforts. 

Issues for Funders and Policymakers 

1 - How to create needed learning products. 

Both developers and implementers need additional learning products in order to 
increase competence in fidelity/adaptation balance. Learning products and related 
items should be created and shared widely with individual practitioners, program 
administrators, policymakers, and researchers as they wrestle with the complex 
issues presented here. 

Products need to cover both basic principles and practical checklists for how to con- 
duct a fidelity/adaptation review. Training programs need to be designed to provide 
hands-on, peer-connected training for using the new information. Finally, these 
learning resources need to be implemented in the field through larger “culture 
change” efforts to encourage the field as a whole to behave differently. These efforts 
will require partnerships with national and regional substance abuse prevention and 
other public health organizations. 

2 - How to provide needed training and technical assistance. 

A training program should be created to support prevention practitioners, program 
administrators, and community leaders in addressing the issues of fidelity/adaptation. 

Technical assistance also needs to be available on a customized basis, preferably 
involving program developers and experienced implementers. Existing systems of 
Federal agencies, such as the CAPT program in CS AP, can help to offer this techni- 
cal assistance. Encouragement, and possibly funding support, can also be offered to 
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Other groups that interface with the community of program implementers to provide 
technical assistance. 

In addition, there should be a program for “culture change” through such strategies as: 

• Leadership statements by key organizations at their conferences, in publica- 
tions, and on Web sites; identifying fidelity/adaptation as an issue the field 
needs to deal with more fully; 

• Assembling “think tanks” to look at motivational, institutional, and political 
factors that might be shaped to support this culture change; and 

• Setting up a larger range of partnerships with other Federal agencies, foun- 
dations, and the private sector. 

3 - Now to shape funding priorities. 

To make a lasting difference in improving the quality of program implementation in 
substance abuse prevention, including the balancing of program fidelity and adapta- 
tion, funding priorities will need to change. Funding will need to be earmarked for 
these activities, both in the grants for implementation programs and by creating new 
sources of funding for technical assistance support and field development work in 
this important area. 

4 - How to promote overall capacity building for the prevention field. 

For both communities and prevention agencies, the resources recommended above 
can only be used fully if they are set in the larger context of efforts to strengthen the 
overall organization. Long-term planning for sustainability, dealing with initial 
needs/assets and readiness assessments, and sensitive balancing of program fidelity 
and adaptation are typical of the complex interventions that cannot be undertaken by 
weak, under-resourced entities. 
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Next Steps 

The Center for Substance Abuse Prevention is already addressing many of the issues 
noted above. For instance, through its National Center for the Advancement of 
Prevention four small-scale efforts to explore unresolved issues in fidelity/adaptation 
balance are in progress: 

1 - The perspectives of program developers on fidelity/adaptation are being obtained 
through discussions with several prominent developers about these issues for their 
own programs, e.g., how these developers deal with enhancements and modifica- 
tions of their programs over time. 

2 - Perspectives of implementers for these same programs are being obtained 
through discussions with selected field implementers, to learn more about exactly 
how fidelity or adaptation challenges are dealt with. 

3 - A review of fidelity/adaptation instruments is being conducted, to lead to infor- 
mation on (a) format and content for fidelity instruments, and how these could be 
better standardized in the substance abuse prevention field; (b) what steps could be 
taken in the field to increase the use of fidelity instruments as measures of imple- 
mentation quality; and (c) how prevention practitioners/agencies and communities 
can use these instruments in practical ways. 

4 - A thought paper is being written on how future progress in dealing with 
fidelity/adaptation balance may require obtaining information on these matters 
prospectively. 

In the end, these and other activities proposed to increase attention to the important 
issue of fidelity and adaptation balance will assist program developers and implementers 
to achieve desired outcomes. For example, CSAP is planning to provide clear guidance, 
including core components analysis, fidelity instruments, and other products to help the 
field ensure adaptation/fidelity balance and maintain the effectiveness of prevention pro- 
grams, maintain accountability, and re-shape the field. 
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As a final recommendation, there should be ongoing exploration of other content areas 
that could provide additional insights about fidelity/adaptation balance and related 
issues. Medication compliance is one area that might be especially fruitful, because that 
field deals with the problem of people not following the medication regimen specified 
for them. Adult education is another study area for concepts in designing implementa- 
tion strategies and fidelity/adaptation balances (McDonald, 2000), since in the end, 
implementation almost always involves changing the behavior of adults — teachers, pre- 
vention personnel, etc. — through an educational process so that they will then imple- 
ment a program properly. 
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Conclusion 

Returning to the initial comparison that implementing a program is like constructing a 
building quoted earlier, the architectural plans represent a degree of purpose in building 
that is similar to substance abuse prevention programming. Program implementation — 
including the balancing of fidelity and adaptation — is most likely to succeed when the 
changes made are highly intentional and rarely when they are accidental or careless. 

The review of literature and conclusions drawn from this effort address the desire of 
those working “in the trenches” of prevention to have usable strategies for balancing 
program fidelity and adaptation, and for dealing with the other stages of the program 
implementation process. 

There is a growing knowledge base on how to do this well. The recommendations made 
here can help both to increase the knowledge and to shape it into learning products and 
interventions that will improve the future quality of implementation practice in sub- 
stance abuse prevention. 
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